Development of Data Quality Indicators for Improving Hospital International Classification of Diseases-Coded Health Data Quality Globally

BACKGROUND

Hospital inpatient data, coded using the International Classification of Diseases (ICD), is widely used to monitor diseases, allocate resources and funding, and evaluate patient outcomes. As such, hospital data quality should be measured before use; however, currently, there is no standard and international approach to assess ICD-coded data quality.

OBJECTIVE

To develop a standardized method for assessing hospital ICD-coded data quality that could be applied across countries: Data quality indicators (DQIs).

RESEARCH DESIGN

To identify a set of candidate DQIs, we performed an environmental scan, reviewing gray and academic literature on data quality frameworks and existing methods to assess data quality. Indicators from the literature were then appraised and selected through a 3-round Delphi process. The first round involved face-to-face group and individual meetings for idea generation, while the second and third rounds were conducted remotely to collect online ratings. Final DQIs were selected based on the panelists' quantitative and qualitative feedback.

SUBJECTS

Participants included international experts with expertise in administrative health data, data quality, and ICD coding.

RESULTS

The resulting 24 DQIs encompass 5 dimensions of data quality: relevance, accuracy and reliability; comparability and coherence; timeliness; and Accessibility and clarity. These will help stakeholders (eg, World Health Organization) to assess hospital data quality using the same standard across countries and highlight areas in need of improvement.

CONCLUSIONS

This novel area of research will facilitate international comparisons of ICD-coded data quality and be valuable to future studies and initiatives aimed at improving hospital administrative data quality.

Development and Validation of Knowledge Assessment Scales for Dementia and Urinary Incontinence in Community Older People

To develop and validate scales for reliably assessing dementia and urinary incontinence knowledge of older adults in the community. Items were generated through a literature review, refined through a Delphi study (n = 19), and then revised through a pilot study (n = 29). Item analysis and exploratory factor analysis were applied to finalize the scales (n = 244). Construct validity, reliability, and acceptability were evaluated (n = 243). The two knowledge assessment scales for dementia and urinary incontinence, respectively, comprised 12 items and 8 items. Model fit indicators of both met the criteria of confirmatory factor analysis. Cronbach's α were .82 and .70, respectively. Completion ratio and completion time of the two scales was 83.51% and 4.22 ± 1.90 minutes. The knowledge assessment scales for dementia and urinary incontinence with satisfactory validity, reliability, and acceptability, could be served as valid tools for disease prevention and management among older adults in the community.

Developing pre-licensure interprofessional and stroke care competencies through skills-based simulations

Interprofessional collaboration (IPC) in stroke care is accepted as best practice and necessary given the multi-system challenges and array of professionals involved. Our two-part stroke team simulations offer an intentional interprofessional educational experience (IPE) embedded in pre-licensure occupational therapy, physical therapy, pharmacy, medicine, nursing and speech-language pathology curricula. This six-year mixed method program evaluation aimed to determine if simulation delivery differences necessitated by COVID-19 impacted students' IPC perception, ratings, and reported learning. Following both simulations, the Interprofessional Collaborative Competency Assessment Scale (ICCAS) and free-text self-reported learning was voluntarily and anonymously collected. A factorial ANOVA using the ICCAS interprofessional competency factors compared scores across delivery methods. Content and category analysis was done for free-text responses. Overall, delivery formats did not affect positive changes in pre-post ICCAS scores. However, pre and post ICCAS scores were significantly different for interprofessional competencies of roles/responsibilities and collaborative patient/family centered approach. Analysis of over 10,000 written response to four open-ended questions revealed the simulation designs evoked better understanding of others' and own scope of practice, how roles and shared leadership change based on context and client need, and the value of each team member's expertise. Virtual-experience-only students noted preference for an in-person stroke clinic simulation opportunity.

The development of recommendations for the assessment and management of sesamoiditis by podiatrists: A Delphi and content validity study

INTRODUCTION

Sesamoiditis is a common, and often painful, musculoskeletal pathology frequently encountered by podiatrists. However, there are currently no recommendations to guide podiatrists in the assessment and management of people with sesamoiditis. The aim of this study was to develop consensus-driven clinical recommendations on the assessment and management of people with sesamoiditis.

METHODS

A four-round online Delphi survey was conducted with a panel of New Zealand and Australian podiatrists. In the first round, panellists answered open-ended questions that were used to create statements. In round two, the panellists scored the statements from 1 to 9 (1 = not at all important, 9 = absolutely essential). Consensus was defined using the RAND/University of California Los Angles Disagreement Index. Panellists were asked to reconsider statements that did not achieve consensus in round three. In the final round, content validity and acceptability of the statements for inclusion in clinical recommendations were determined using content validity ratios and the Content Validity Index (CVI).

RESULTS

Eighteen panellists completed round one with 16 (89%) completing all four rounds. A total of 118 statements were generated following round one. Following rounds two and three, 78 statements were accepted by panellists as being important, with 62 statements achieving sufficient content validity for inclusion in clinical recommendations. The CVI for these 62 statements was 0.58. These recommendations provide guidance on subjective assessment (pain characteristics/symptomology, activity/sports/training history and medical history) objective assessment (establishing a diagnosis, identifying contributing biomechanical factors, footwear/orthoses, ruling out differential diagnoses) and management (temporary padding/strapping, education, footwear, foot orthoses and when to consider referral).

CONCLUSION

This consensus exercise has provided a set of consensus-based recommendations for the assessment and management of people with sesamoiditis. In the current absence of research-based evidence in this area, these recommendations are intended to support clinicians. The recommendations may also serve as a basis for future clinical trials evaluating the efficacy of conservative interventions for people with sesamoiditis.

Competencies for Those Who Coach Physicians: A Modified Delphi Study

The rapidly evolving coaching profession has permeated the health care industry and is gaining ground as a viable solution for addressing physician burnout, turnover, and leadership crises that plague the industry. Although various coach credentialing bodies are established, the profession has no standardized competencies for physician coaching as a specialty practice area, creating a market of aspiring coaches with varying degrees of expertise. To address this gap, we employed a modified Delphi approach to arrive at expert consensus on competencies necessary for coaching physicians and physician leaders. Informed by the National Board of Medical Examiners' practice of rapid blueprinting, a group of 11 expert physician coaches generated an initial list of key thematic areas and specific competencies within them. The competency document was then distributed for agreement rating and comment to over 100 stakeholders involved in physician coaching. Our consensus threshold was defined at 70% agreement, and actual responses ranged from 80.5% to 95.6% agreement. Comments were discussed and addressed by 3 members of the original group, resulting in a final model of 129 specific competencies in the following areas: (1) physician-specific coaching, (2) understanding physician and health care context, culture, and career span, (3) coaching theory and science, (4) diversity, equity, inclusion, and other social dynamics, (5) well-being and burnout, and (6) physician leadership. This consensus on physician coaching competencies represents a critical step toward establishing standards that inform coach education, training, and certification programs, as well as guide the selection of coaches and evaluation of coaching in health care settings.

An mHealth application for chronic vascular access: Consumer led co-creation

PURPOSE

Children with chronic and complex health conditions frequently need intravenous devices. The current approach to intravenous device selection, insertion, and monitoring is inconsistent, and healthcare consumers are often negatively affected by siloed health information, and poor future planning. Despite child- and family-centred care being recognised as a pillar of paediatric nursing care, limited implementation for vascular access device planning and management is evident.

DESIGN AND METHODS

To address this, we conducted a multi-phased approach to co-create, then evaluate, a mobile health (mHealth) application: IV Passport. Co-creation involved a prioritisation survey, followed by a Passport advisory panel consensus meeting. Following confirmation of the required content and features of the Passport, the mHealth application was designed and content validation achieved via survey.

RESULTS

The prioritisation survey yielded recommendations for seven features (e.g., graphical presentations of current/past devices). Content for nine device types (e.g., totally implanted ports) was suggested, each with 10 related items (e.g., insertion site). Content items for device-associated complications, future vascular access plans, and educational resources were also suggested. Following design, the application was released through Apple and Android platforms; and adapted to a paper version. Content validation was established; 100% strongly agreed the application was easy to use; 80% agreed/strongly agreed that they would recommend the Passport to others.

CONCLUSION

IV Passport embodies effective child- and family-centred care through consumer co-creation to empower patients and families manage vascular access devices.

PRACTICE IMPLICATIONS

IV Passport remains active; and can be utilised across many healthcare settings and patient populations.

Development of consensus on essential virtues for ethics and research integrity training using a modified Delphi approach

Most ethics and research integrity (ERI) training approaches are based on teaching moral rules, duties or responsibilities, often not sufficiently addressing virtue-based ethics. This study aimed to obtain a consensus among relevant experts on the importance of essential virtues for ERI training and their acquisitions. A modified Delphi consensus process was conducted in three rounds; 31 ERI experts participated in Round 1 and 23 in Round 2 and Round 3. Based on findings generated from qualitative data in Round 1, a structured questionnaire with 90 different statements grouped under five domains was developed for Round 2 and Round 3. After the final round, a consensus was achieved on two-thirds of statements included in this study. The experts agreed that virtues are based on learned and reflected attitudes and that the appropriate direction to acquire research virtues is through continuing education using case studies and discussions based on real-life scenarios. Furthermore, the consensus was obtained on 35 scientific virtues that should be stimulated in ERI training, prioritizing honesty, integrity, accountability, criticism and fairness as the most essential scientific virtues for good research practice. These results should be considered in developing or adjusting the ERI training program and materials.

Advancing health equity for Indigenous peoples in Canada: development of a patient complexity assessment framework

BACKGROUND

Indigenous patients often present with complex health needs in clinical settings due to factors rooted in a legacy of colonization. Healthcare systems and providers are not equipped to identify the underlying causes nor enact solutions for this complexity. This study aimed to develop an Indigenous-centered patient complexity assessment framework for urban Indigenous patients in Canada.

METHODS

A multi-phased approach was used which was initiated with a review of literature surrounding complexity, followed by interviews with Indigenous patients to embed their lived experiences of complexity, and concluded with a modified e-Delphi consensus building process with a panel of 14 healthcare experts within the field of Indigenous health to identify the domains and concepts contributing to health complexity for inclusion in an Indigenous-centered patient complexity assessment framework. This study details the final phase of the research.

RESULTS

A total of 27 concepts spanning 9 domains, including those from biological, social, health literacy, psychological, functioning, healthcare access, adverse life experiences, resilience and culture, and healthcare violence domains were included in the final version of the Indigenous-centered patient complexity assessment framework.

CONCLUSIONS

The proposed framework outlines critical components that indicate the presence of health complexity among Indigenous patients. The framework serves as a source of reference for healthcare providers to inform their delivery of care with Indigenous patients. This framework will advance scholarship in patient complexity assessment tools through the addition of domains not commonly seen, as well as extending the application of these tools to potentially mitigate racism experienced by underserved populations such as Indigenous peoples.

Initiatives for promoting educator wellbeing: a Delphi study

BACKGROUND

Growing demands on educators have resulted in increased levels of burnout and stress and decreasing wellbeing. This study aimed to establish expert consensus on the key characteristics required in prospective educator wellbeing initiatives.

METHODS

The Delphi approach is a process of forecasting that is based on the aggregated opinion of panel members (or experts) within a field of study. Using a Delphi approach, academic and practitioner expertise were sought over a two-rounds, with 17 and 14 participants in each round respectively. The study aimed to identify how systemic factors (e.g., leadership) could be utilised to promote educator wellbeing. The study also sought expert consensus on enablers and barriers for engagement in educator wellbeing initiatives.

RESULTS

Findings highlighted the importance of fostering positive relationships with colleagues, communities and families, and the active role of wellbeing teams to promote wellbeing initiatives. The need for leaders to address their own wellbeing and build trust within teams was also identified. Panel members identified the need for prospective funding to prioritise wellbeing initiatives. There was also a preference for ongoing initiatives rather than stand-alone wellbeing events that conveyed the ongoing importance of managing one's wellbeing.

CONCLUSIONS

This paper presents practical recommendations that can be used to inform the development and evaluation of future initiatives and policy. Applying the consensus derived from this study is likely to make wellbeing initiatives more viable and facilitate uptake amongst educators.

Framework Development: Standardizing Definition of Advanced Practice Radiation Therapy Activities for Clinical Workload Quantification

Purpose

Advanced practice (AP) in radiation therapy (RT) is being implemented around the globe. In an effort to advance the understanding of the similarities and differences in APRT roles in Ontario, Canada, a community of practice (CoP) sought ways to provide quantitative data on the nature of APRT clinical activities and the frequency with which these activities were being executed.

Methods

In 2017, a consensus building project involving 20 APRTs and 14 radiation therapy (RT) department managers in Ontario was completed to establish a mechanism to quantify APRTs' clinical impact. In Round 1 & 2, expert feedback was gathered to generate an Advanced Practice (AP) Activity List. In Round 3: 20 APRTs completed an online survey to assess the importance and applicability of each AP Activity to their role using Likert scale (0-5). A final AP Activity List & Definitions was generated.

Results & discussion

Round 1: Forty-seven AP activities were identified. Round 2: 3/14 RT managers provided 145 feedback statements on Round 1 AP Activity List. The working group used RT managers' feedback to clarify AP activities and definitions, specifically merging 33 unique AP activities to create 11 inclusive AP activities and eliminating 8 activities identified from Round 1. The most inclusive AP activity created was #1 New Patient Consultation, this AP Activity is merged from 7 unique AP activities. Incorporating RT managers' feedback with the internal AP clinical workload lists from 2 Ontario cancer centres resulted in a revised AP Activity List with 20 AP inclusive activities. Round 3: 14/20 APRTs provided Likert scores on this revised list. The most applicable AP activities (mean score) were #16 Technical Consultation (4.0), #15 Contouring Target Volume (3.8) and #2 Planning Consultation (3.8); the least applicable was #18 MR Applicator Assessment (0.9).

Conclusions

This is the first systematic attempt to build consensus on AP clinical activities. Non-clinical APRT activities related to research, education, innovation, and program development were not in the scope of this project. The Final AP Activity List & Definitions serves as a framework that allows standardized and continuous monitoring of AP clinical activities and impact.

Developing an instructional module of 'digital dentistry' for undergraduate prosthodontics curriculum

Purpose

To prepare a module on 'Digital Dentistry", which can be used to sensitize dental undergraduate students during their final year Prosthodontics curriculum.

Methods

Ethical approval (167/Ethics/2022) was taken from the Institution before starting the study. Need assessment was achieved using focused group discussion (Online Zoom meeting) of various stakeholders of the Department of Prosthodontics of the institution. A preliminary module content was prepared in google questionnaire form after thorough literature review by the research team and final content was developed using modified Delphi technique in 3 questionnaire rounds with the help of 27 experts.

Result

Thematic analysis of Need Assessment focussed group discussion led to the emergence of both theoretical and practical aspect of learning in digital dentistry. The final module was developed with the process followed in modified Delphi technique.

Conclusion

Within the limitations of the study the module of Digital dentistry was developed which can be implemented in most of the dental institutes depending on the resources and available infrastructure.

Evaluating consistency of physical activity and exercise prescription in the UK for people with diabetes - a Delphi study

Introduction

Increased physical activity is recommended as a cost-effective measure to tackle long-term management of people with diabetes, but research on interventions lacks consistency in terms of effective duration and modality.

Methods

The aim of this study was to evaluate expert consensus on exercise and physical activity prescription via a three-round Delphi study conducted with 45 UK-based health and fitness professionals experienced in prescribing exercise or physical activity to people with diabetes.

Results

The majority of items put forward to the panel reached consensus with 70% or above voting these items as important, but the details of the type, duration and/or modality of exercise or physical activity prescription within these items often contradicted each other, suggesting that patients are receiving inconsistent advice. The range of different exercise prescription found in this study suggests that patients are being given inconsistent and potentially confusing advice, which may affect their participation in exercise and long-term lifestyle change.

Conclusion

More consistent promotion of advice from healthcare and fitness professionals may help with increasing physical activity in this participant group and achieving long term behavior change, reducing patient symptoms as well as reducing the cost to the National Health Service (NHS).

Preferred Reporting Items for Resistance Exercise Studies (PRIRES): A Checklist Developed Using an Umbrella Review of Systematic Reviews

BACKGROUND

The issues of replication and scientific transparency have been raised in exercise and sports science research. A potential means to address the replication crisis and enhance research reliability is to improve reporting quality and transparency. This study aims to formulate a reporting checklist as a supplement to the existing reporting guidelines, specifically for resistance exercise studies.

METHODS

PubMed (which covers Medline) and Scopus (which covers Medline, EMBASE, Ei Compendex, World Textile Index, Fluidex, Geobase, Biobase, and most journals in Web of Science) were searched for systematic reviews that comprised the primary studies directly comparing different resistance training methods. Basic data on the selected reviews, including on authors, publication years, and objectives, were summarized. The reporting items for the checklist were identified based on the objective of the reviews. Additional items from an existing checklist, namely the Consensus on Exercise Reporting Template, a National Strength and Conditioning Association handbook, and an article from the EQUATOR library were incorporated into the final reporting checklist.

RESULTS

Our database search retrieved 3595 relevant records. After automatic duplicate removal, the titles and abstracts of the remaining 2254 records were screened. The full texts of 137 records were then reviewed, and 88 systematic reviews that met the criteria were included in the umbrella review.

CONCLUSION

Developed primarily by an umbrella review method, this checklist covers the research questions which have been systematically studied and is expected to improve the reporting completeness of future resistance exercise studies. The PRIRES checklist comprises 26 reporting items (39 subitems) that cover four major topics in resistance exercise intervention: 1) exercise selection, performance, and training parameters, 2) training program and progression, 3) exercise setting, and 4) planned vs actual training. The PRIRES checklist was designed specifically for reporting resistance exercise intervention. It is expected to be used with other reporting guidelines such as Consolidated Standards of Reporting Trials and Standard Protocol Items: Recommendations for Interventional Trials. This article presents only the development process and resulting items of the checklist. An accompanying article detailing the rationale for, the importance of, and examples of each item is being prepared.

REGISTRATION

This study is registered with the EQUATOR Network under the title "Preferred Reporting Items for Resistance Exercise Studies (PRIRES)." PROSPERO registration number: CRD42021235259.

Management of amyotrophic lateral sclerosis in clinical practice: Results of the expert consensus using the Delphi methodology

Amyotrophic lateral sclerosis (ALS) is a rare disease characterized by a progressive and irreversible degeneration of upper and lower motor neurons leading to death. In France, limited data exist describing the criteria used in clinical practice for diagnosis and follow-up, and how novel therapies may fit in. The objective of this Delphi panel was to obtain an overview of current French practices in ALS diagnosis, management, and follow-up by determining the scales and criteria used in clinical practice outside of clinical trials, as well as the place of a future treatment like AMX0035, acting on endoplasmic reticulum (ER) stress and mitochondrial dysfunction, in the current therapeutic strategies. A questionnaire was administered to 24 ALS healthcare providers practicing in ALS centers in France. Two rounds of remote voting were organized, before proposition of final consensus statements. Consensus was considered reached when at least 66% of the voters agreed. Consensus were obtained to define the new Gold Coast criteria as the ones used in clinical practice to establish the diagnosis of ALS, thus replacing the revised El Escorial criteria, considered too complex and now mainly used to characterize the patient populations to be included in clinical trials. The clinical factors considered to establish ALS diagnosis are mainly the demonstration of progression of the motor deficit and elimination of differential diagnoses. The ALSFRS-R scale is used in daily clinical practice to assess patient's functional impairment in terms of number of points lost, with the bulbar, respiratory, and fine motor subscores being the most important to evaluate independently. A critical medical need was identified regarding the provision of new therapeutic alternatives in ALS. The panel members would support the earliest management of patients. In this landscape, based on data from a very encouraging phase II (Centaur trial), AMX0035 represents a new tool of choice in current treatment strategies for all patients for whom experts are confident in the diagnosis of ALS, in combination with riluzole. These results will need to be confirmed by the ongoing phase III trial (Phoenix trial).

Why do some people who had a stroke not receive the recommended 45 min of occupational therapy and physiotherapy? A Delphi study

OBJECTIVES

To gain a consensus among therapists for reasons why a person who had a stroke may not receive the Royal College of Physicians' recommended minimum of 45 min of daily therapy.

DESIGN

Three-round remote e-Delphi study.

SETTING

National study, based in the UK.

PARTICIPANTS

Occupational therapists and physiotherapists with experience of delivering therapy after stroke and awareness of the 45 min guideline.

RESULTS

Forty-five therapists consented to participate in the study. Thirty-five (78%) completed round one, 29 of 35 (83%) completed round 2 and 26 of 29 (90%) completed round three. A consensus (75%) was reached for 32 statements. Reasons why a person may not receive 45 min were related to the suitability of the guideline for the individual (based on factors like therapy tolerance or medical status) or the capability of the service to provide the intervention. In addition to the statements for which there was a consensus, 32 concepts did not reach a consensus. Specifically, there was a lack of consensus concerning the suitability of the guideline for people receiving Early Supported Discharge (ESD) services and a lack of agreement about whether people who need more than 45 min of therapy actually receive it.

CONCLUSION

Some people do not receive 45 min of therapy as they are considered unsuitable for it and some do not receive it due to services' inability to provide it. It is unclear which reasons for guideline non-achievement are most common. Future research should focus on why the guideline is not achieved in ESD, and why people who require more than 45 min may not receive it. This could contribute to practical guidance for therapists to optimise therapy delivery for people after stroke.

Revising the Pragmatics Profile using a modified Delphi methodology to meet the assessment needs of current speech-language therapists

BACKGROUND

Non-standardized assessment tools are preferred when assessing communication of individuals with developmental disabilities. Currently, there are limited tools available for assessing this population. Informant report tools such as the Pragmatics Profile (PP) of Everyday Communication Skills are beneficial in gathering a representative view of an individual's communication. However, the PP is out of print and outdated, requiring revisions to meet contemporary assessment needs of speech-language therapists (SLTs).

AIMS

To seek consensus from an international panel regarding revising the Pragmatic Profile by (1) updating language and terminology, and (2) development of an online tool.

METHODS & PROCEDURES

A total of 13 experienced SLTs and researchers in the disability field participated in a modified Delphi study including an initial online meeting followed by an anonymous four-round survey. Participants reviewed the relevance and wording of questions in the original preschool, school-age and adult versions to create a single combined version of the PP. In each Delphi round, the level of consensus was calculated and qualitative comments were analysed using thematic analysis.

OUTCOMES & RESULTS

A revised online version of the PP was created including 64 questions. Qualitative analysis illuminated key concepts in the creation of a revised form including the need for plain and age-neutral language, which is inclusive of all communication modalities and physical impairments, and identifies behaviours that have the potential to be communicative acts. Using conditional logic, users are navigated to the appropriate questions based on the intentionality level of the individual rather than their age.

CONCLUSIONS & IMPLICATIONS

This study resulted in the revision of a valued assessment tool appropriate for current disability service provision that identifies communication along the continuum of intentionality rather than age.

WHAT THIS PAPER ADDS

What is already known on this subject Non-standardized tools are appropriate when assessing communication of individuals with developmental disabilities. However, there are limited published tools suitable for this population with several of them out of print, making it difficult to conduct a holistic assessment. What this study adds to the existing knowledge This study resulted in the creation of an online PP based on experts' opinion. The revised PP modified the primary focus of the tool from age- to skill-based whereby questions are targeted according to intentionality level. Revisions included plain language, and inclusion of all communication modalities and physical impairments via a series of prompts to ensure that the information provided by informants is accurate and relevant. What are the potential or actual clinical implications of this work? The revised PP adds to the toolkit of an SLT working with individuals with a developmental disability and allows for accurate reporting of functional communication. Guided by experts' opinion, the revised PP is likely to be highly valued in the increasingly technological world in which we live.

Diagnosis of vocal cord dysfunction/inducible laryngeal obstruction: An International Delphi Consensus Study

BACKGROUND

Vocal cord dysfunction/inducible laryngeal obstruction (VCD/ILO) is characterized by breathing difficulties in association with excessive supraglottic or glottic laryngeal narrowing. The condition is common and can occur independently; however, it may also be comorbid with other disorders or mimic them. Presentations span multiple specialties and misdiagnosis or delayed diagnosis is commonplace. Group-consensus methods can efficiently generate internationally accepted diagnostic criteria and descriptions to increase clinical recognition, enhance clinical service availability, and catalyze research.

OBJECTIVES

We sought to establish consensus-based diagnostic criteria and methods for VCD/ILO.

METHODS

We performed a modified 2-round Delphi study between December 7, 2021, and March 14, 2022. The study was registered at ANZCTR (Australian New Zealand Clinical Trials Registry; ACTRN12621001520820p). In round 1, experts provided open-ended statements that were categorized, deduplicated, and amended for clarity. These were presented to experts for agreement ranking in round 2, with consensus defined as ≥70% agreement.

RESULTS

Both rounds were completed by 47 international experts. In round 1, 1102 qualitative responses were received. Of the 200 statements presented to experts across 2 rounds, 130 (65%) reached consensus. Results were discussed at 2 international subject-specific conferences in June 2022. Experts agreed on a diagnostic definition for VCD/ILO and endorsed the concept of VCD/ILO phenotypes and clinical descriptions. The panel agreed that laryngoscopy with provocation is the gold standard for diagnosis and that ≥50% laryngeal closure on inspiration or Maat grade ≥2 define abnormal laryngeal closure indicative of VCD/ILO.

CONCLUSIONS

This Delphi study reached consensus on multiple aspects of VCD/ILO diagnosis and can inform clinical practice and facilitate research.

Establishing consensus for the core concepts of physiology in the Australian higher education context using the Delphi method

A set of core concepts ("big ideas") integral to the discipline of physiology are important for students to understand and demonstrate their capacity to apply. We found poor alignment of learning outcomes in programs with physiology majors (or equivalent) from 17 Australian universities and the 15 core concepts developed by a team in the United States. The objective of this project was to reach Australia-wide consensus on a set of core concepts for physiology, which can be embedded in curricula across Australian universities. A four-phase Delphi method was employed, starting with the assembling of a Task Force of physiology educators with extensive teaching and curriculum development expertise from 25 Australian universities. After two online meetings and a survey, the Task Force reached agreement on seven core concepts of physiology and their descriptors, which were then sent out to the physiology educator community across Australia for agreement. The seven core concepts and their associated descriptions were endorsed through this process (n = 138). In addition, embedding the core concepts across the curriculum was supported by both Task Force members (85.7%) and educators (82.1%). The seven adopted core concepts of human physiology were Cell Membrane, Cell-Cell Communication, Movement of Substances, Structure and Function, Homeostasis, Integration, and Physiological Adaptation. The core concepts were subsequently unpacked into themes and subthemes. If adopted, these core concepts will result in consistency across curricula in undergraduate physiology programs and allow for future benchmarking.NEW & NOTEWORTHY This is the first time Australia-wide agreement has been reached on the core concepts of physiology with the Delphi method. Embedding of the core concepts will result in consistency in physiology curricula, improvements to teaching and learning, and benchmarking across Australian universities.

Rating versus ranking in a Delphi survey: a randomized controlled trial

BACKGROUND

The Delphi technique has steeply grown in popularity in health research as a structured approach to group communication process. Rating and ranking are two different procedures commonly used to quantify participants' opinions in Delphi surveys. We explored the influence of using a rating or ranking approach on item prioritization (main outcome), questionnaire completion time, and evaluation of task difficulty in a Delphi survey aimed at identifying priorities for the organization of primary cardiovascular care.

METHODS

A randomized controlled parallel group trial was embedded in a three-round online Delphi survey. After an "open" first round, primary care patients, trained patient partners, and primary care clinicians from seven primary care practices were allocated 1:1 to a rating or ranking assessment group for the remainder of the study by stratified permuted block randomization, with strata based on participants' gender and status. Agreement on item prioritization between the experimental groups was measured by calculating Krippendorff's alpha reliability coefficient on the aggregate rank order of items in each group after the final round. Self-reported ease or difficulty with the assessment task was measured with the Single Ease Question.

RESULTS

Thirty-six panelists (13 clinic patients, 7 patient partners, 16 clinicians; 60% females) were randomized to the rating (n = 18) or ranking (n = 18) group, with 30 (83%) completing all rounds. Both groups identified the same highest priorities from a set of 41 items, but significant discrepancies were found as early as the seventh top item. There was moderately strong agreement between the priority ordering of top items common to both groups (Krippendorff's alpha = 0.811, 95% CI = 0.669-0.920). A 9-min mean difference to complete the third-round questionnaire in favor of the rating group failed to achieve statistical significance (p = 0.053). Ranking was perceived as more difficult (p < 0.001).

CONCLUSIONS

A rating or ranking procedure led to modestly similar item prioritization in a Delphi survey, but ranking was more difficult. This study should be replicated with a larger number of participants and with variations in the ranking and rating procedures.

TRIAL REGISTRATION

Not applicable.

Transfusion medicine curricular content for general pediatricians and pediatric subspecialists: A national multi-specialty Delphi consensus study

BACKGROUND

Although pediatric residents frequently order blood products, transfusion medicine (TM) education is both limited and unstandardized during postgraduate training. Using Delphi methodology, this study aimed to identify and prioritize which pediatric TM curricular topics are most important to inform postgraduate training in TM for general pediatricians and pediatric subspecialists.

METHODS

A national panel of experts iteratively rated potential curricular topics, on a 5-point scale, to determine their priority for inclusion within a TM curriculum. After each round, responses were analyzed. Topics receiving a mean rating <3/5 were removed from subsequent rounds and remaining topics were resent to the panel for further ratings until consensus was achieved, defined as Cronbach α ≥ 0.95. At conclusion of the Delphi process, topics rated ≥4/5 were considered core curricular topics, while topics rated ≥3 to <4 were considered extended topics.

RESULTS

Forty-five TM experts from 17 Canadian institutions and 12 subspecialties completed the first Delphi round and 31 completed the second. Fifty-seven potential curricular topics were generated from a systematic literature review and Delphi panelists. Two survey rounds were completed before consensus was achieved. Seventy-three topics in six domains reached consensus: 31 core curricular topics and 42 extended topics. There were no significant differences in ratings between TM and non-TM specialists.

DISCUSSION

A multispecialty Delphi panel reached consensus in identification of curricular topics for pediatric resident physicians. These results set the stage to develop a pediatric TM curriculum that will be foundational for pediatric trainees to enhance learning and improve transfusion safety.

Forming a consensus opinion to inform long COVID support mechanisms and interventions: a modified Delphi approach

Background

Current approaches to support patients living with post-COVID condition, also known as Long COVID, are highly disparate with limited success in managing or resolving a well-documented and long-standing symptom burden. With approximately 2.1 million people living with the condition in the UK alone and millions more worldwide, there is a desperate need to devise support strategies and interventions for patients.

Methods

A three-round Delphi consensus methodology was distributed internationally using an online survey and was completed by healthcare professionals (including clinicians, physiotherapists, and general practitioners), people with long COVID, and long COVID academic researchers (round 1 n = 273, round 2 n = 186, round 3 n = 138). Across the three rounds, respondents were located predominantly in the United Kingdom (UK), with 17.3-15.2% (round 1, n = 47; round 2 n = 32, round 2 n = 21) of respondents located elsewhere (United States of America (USA), Austria, Malta, United Arab Emirates (UAE), Finland, Norway, Malta, Netherlands, Iceland, Canada, Tunisie, Brazil, Hungary, Greece, France, Austrailia, South Africa, Serbia, and India). Respondents were given ∼5 weeks to complete the survey following enrolment, with round one taking place from 02/15/2022 to 03/28/22, round two; 05/09/2022 to 06/26/2022, and round 3; 07/14/2022 to 08/09/2022. A 5-point Likert scale of agreement was used and the opportunity to include free text responses was provided in the first round.

Findings

Fifty-five statements reached consensus (defined as >80% agree and strongly agree), across the domains of i) long COVID as a condition, ii) current support and care available for long COVID, iii) clinical assessments for long COVID, and iv) support mechanisms and rehabilitation interventions for long COVID, further sub-categorised by consideration, inclusion, and focus. Consensus reached proposes that long COVID requires specialised, comprehensive support mechanisms and that interventions should form a personalised care plan guided by the needs of the patients. Supportive approaches should focus on individual symptoms, including but not limited to fatigue, cognitive dysfunction, and dyspnoea, utilising pacing, fatigue management, and support returning to daily activities. The mental impact of living with long COVID, tolerance to physical activity, emotional distress and well-being, and research of pre-existing conditions with similar symptoms, such as myalgic encephalomyelitis, should also be considered when supporting people with long COVID.

Interpretation

We provide an outline that achieved consensus with stakeholders that could be used to inform the design and implementation of bespoke long COVID support mechanisms.

Funding

None.

Consensus-based statements for assessing clinical competency in podiatry-related work integrated learning

BACKGROUND

The training of undergraduate and graduate-entry podiatry students in Australia and New Zealand includes practical sessions in a simulated and real-life clinical setting and Work Integrated Learning (WIL) comprising professional clinical placements. Student performance during WIL is evaluated by their Clinical Educators using clinical competency tools. Having a standardised and validated clinical assessment tool for WIL in podiatry would facilitate consistency in assessment, promote standardisation between programs, and ensure that all podiatry students are assessed against a set of criteria over the course of their clinical programs to the point of threshold clinical competency. Therefore, the aim of this study was to develop a series of consensus-based statements via Delphi technique as the first step towards developing guidelines to direct the assessment of podiatry students during WIL.

METHODS

This study used a three-round modified Delphi consensus method. A panel of 25 stakeholders was sought. Specifically, representation from each of the universities in Australia and New Zealand who provide entry level programs, Clinical Educators, podiatry student representatives, new podiatry graduates and consumers (podiatrists hiring new graduates). The survey for Round 1 aimed for consensus and consisted of five open-ended questions. Questions one to three asked respondents to nominate what they considered were the important elements that needed to be assessed for podiatry students undertaking WIL for: Clinical performance/skills, Communication and Professional behaviour, Question 4 asked respondents to identify further/other elements of importance, whilst Question 5 asked a) how these elements should be evaluated and b) how should overall competency and ability to progress within the program be determined. Round 2 and 3 aimed to gather agreement and the questions were based on the responses from previous rounds.

RESULTS

Twenty-five participants agreed to participate, 17 females (68%) and eight males (32%). The panel consisted of 10 podiatry educators (40%), nine Clinical Educators (36%), two student representatives (8%), two new podiatry graduates (8%) and two consumers (8%). From the 25 recruited participants, 21 responded to Round one, 18 to Round two and 17 in Round three. At the conclusion of the Delphi survey, 55 statements had reached consensus or agreement.

CONCLUSIONS

This Delphi study is the first of its kind for the podiatry profession to develop consensus-based statements regarding the assessment of WIL. Fifty-five statements pertinent to the assessment of WIL were identified. This is an important first step toward the development of a consistent WIL assessment tool which may be applied across entry-level podiatry programs across Australia and New Zealand.

Consensus on Feedback Statements That Create Responsibility Among People With Parkinson's Disease: A Delphi Study

IMPORTANCE

Sense of agency is associated with a sense of responsibility, which is essential to performing goal-directed occupations.

OBJECTIVE

To reach consensus on a set of extrinsic feedback statements that have the potential to create a sense of responsibility among patients with neurological disorders in the course of performing daily or social occupations.

DESIGN

Anonymous Delphi study with two rounds with international experts and one round with Irani patients with Parkinson's disease (PD).

SETTING

Electronic survey.

PARTICIPANTS

One hundred experts and 73 patients with idiopathic PD.

OUTCOMES AND MEASURES

Experts and patients anonymously rated (5 = strongly agree/very effective, 4 = agree/effective, 3 = neither agree nor disagree/uncertain, 2 = disagree/ineffective, 1 = strongly disagree/very ineffective) their level of agreement with each survey statement and the effectiveness of each statement in creating a sense of responsibility in the course of performing daily or social occupations. Consensus was set as an interquartile range of ≤1 and ≥70% agreement in two adjacent categories of a Likert scale.

RESULTS

In the experts' first round, consensus was reached on the level of agreement and effectiveness of 18 statements. In the second round, final consensus was achieved on all statements. In the one patient round, patients reached consensus on all statements. Finally, 34 statements were rated as 4 or 5 in terms of agreement and effectiveness, based on the opinions of experts and patients.

CONCLUSIONS AND RELEVANCE

This study has produced a collection of feedback statements that might be useful in occupation-based interventions. What This Article Adds: Extrinsic responsibility feedback delivered while administering occupation-based interventions may increase volition, motivation, and engagement.

The new timing in acute care surgery (new TACS) classification: a WSES Delphi consensus study

BACKGROUND

Timely access to the operating room for emergency general surgery (EGS) indications remains a challenge across the globe, largely driven by operating room availability and staffing constraints. The "timing in acute care surgery" (TACS) classification was previously published to introduce a new tool to triage the timely and appropriate access of EGS patients to the operating room. However, the clinical and operational effectiveness of the TACS classification has not been investigated in subsequent validation studies. This study aimed to improve the TACS classification and provide further consensus around the appropriate use of the new TACS classification through a standardized Delphi approach with international experts.

METHODS

This is a validation study of the new TACS by a selected international panel of experts using the Delphi method. The TACS questionnaire was designed as a web-based survey. The consensus agreement level was established to be ≥ 75%. The collective consensus agreement was defined as the sum of the percentage of the highest Likert scale levels (4-5) out of all participants. Surgical emergency diseases and correlated clinical scenarios were defined for each of the proposed classes. Subsequent rounds were carried out until a definitive level of consensus was reached. Frequencies and percentages were calculated to determine the degree of agreement for each surgical disease.

RESULTS

Four polling rounds were carried out. The new TACS classification provides 6 colour-code classes correlated to a precise timing to surgery, defined scenarios and surgical condition. The WHITE colour-code class was introduced to rapidly (within a week) reschedule cancelled or postponed surgical procedures. Haemodynamic stability is the main tool to stratify patients for immediate surgery or not in the presence of sepsis/septic shock. Fifty-one surgical diseases were included in the different colour-code classes of priority.

CONCLUSION

The new TACS classification is a comprehensive, simple, clear and reproducible triage system which can be used to assess the severity of the patient and the surgical disease, to reduce the time to access to the operating room, and to manage the emergency surgical patients within a "safe" timeframe. By including well-defined surgical diseases in the different colour-code classes of priority, validated through a Delphi consensus, the new TACS improves communication among surgeons, between surgeons and anaesthesiologists and decreases conflicts and waste and waiting time in accessing the operating room for emergency surgical patients.

Assessment and Conceptualization of Perceived Competence in Ice Hockey: A Scale Development and Validation Study

Perceived competence plays a crucial role in establishing environments favorable to individual development in youth sport. As most assessment tools of perceived competence are not sport-specific, they have limited usefulness for sport practitioners and researchers. This study had two-fold aims: (i) to develop a perceived competence assessment tool specific to ice hockey; and (ii) to examine its factorial structure and internal consistency. We first developed an initial 29-item version of this ice hockey competence self-report scale, using a group of ice hockey stakeholders and sports science experts to develop the items and a pilot sample of 42 hockey players to confirm test-retest reliability. Finally, we validated the scale among a cohort of 770 adolescent ice hockey players (M age = 14.78, SD = 1.60 years). Results from exploratory factor analysis (EFA) revealed that perceived competence in ice hockey was defined by six dimensions, with rejection of seven items. Confirmatory factor analysis (CFA) suggested that the 6-factor first order model was the best fit with the conceptualization of perceived competence in ice hockey (CFI = 0.938, RMSEA = 0.044). The final, 22-item questionnaire now provides a reliable and valid measure of perceived competence in hockey for adolescent participants. It has promise for evaluating future interventions aiming to foster young athletes' perceived confidence through sport.

Expert Consensus on the Resource Needs of Autistic Children and Young People in Ireland During the Early Months of the COVID-19 Pandemic

The COVID-19 pandemic has brought significant changes across society. This Delphi study aimed to gain expert consensus on challenges faced and resource needs for autistic children during the COVID-19 pandemic. Round 1 of the Delphi method employed semi-structured interviews with experts (N = 24) which were thematically analysed in order to identify needs, resource targets and resource development. In a follow-up Round 2 survey participants rated emergent need and resource in order of priority. Round 2 provided consensus on challenges faced with anxiety, routine and wellbeing ranked most important. Direction on resource design was also obtained. Consensus on the challenges and resources was achieved and is being integrated into a needs-based transition resources toolkit. Future studies could make use of the Delphi method to quickly gain consensus on focus of needs in other contexts and communities.

Expert Consensus on the Characteristics of Patients with Epstein-Barr Virus-Positive Post-Transplant Lymphoproliferative Disease (EBV+ PTLD) for Whom Standard-Dose Chemotherapy May be Inappropriate: A Modified Delphi Study

INTRODUCTION

Following hematopoietic stem cell transplantation or solid organ transplantation, patients are at risk of developing Epstein-Barr virus-positive post-transplant lymphoproliferative disease (EBV⁺ PTLD), which is an ultra-rare and potentially lethal hematologic malignancy. Common treatments for EBV⁺ PTLD include rituximab alone or combined with chemotherapy. Given specific considerations for this population, including severity of the underlying condition requiring transplant, the rigors of the transplant procedure, as well as risks to the transplanted organ, there is a group of patients with EBV⁺ PTLD for whom chemotherapy may be inappropriate; however, there is limited information characterizing these patients. This study aimed to reach expert consensus on the key characteristics of patients for whom chemotherapy may be inappropriate in a real-world setting.

METHODS

A two-round modified Delphi study was conducted to reach consensus among clinicians with expertise treating EBV⁺ PTLD. Articles identified in a targeted literature review guided the development of round 1 and 2 topics and related statements. The consensus threshold for round 1 statements was 75.0%. If consensus was achieved in round 1, the statement was not discussed further in round 2. The consensus thresholds for round 2 were moderate (62.5-75.0%), strong (87.5%), or complete (100.0%).

RESULTS

The panel was composed of a total of eight clinicians (seven hematologists/hemato-oncologists) from six European countries. The panel generated a final list of 43 consensus recommendations on the following topics: terminology used to describe patients for whom chemotherapy may be inappropriate; demographic characteristics; organ transplant characteristics; comorbidities that preclude the use of chemotherapy; EBV⁺ PTLD characteristics; and factors related to treatment-related mortality and morbidity.

CONCLUSIONS

This modified Delphi panel successfully achieved consensus on key topics and statements that characterized patients with EBV⁺ PTLD for whom chemotherapy may be inappropriate. These recommendations will inform clinicians and aid in the treatment of EBV⁺ PTLD.

Use of Delphi in health sciences research: A narrative review

The use of the Delphi technique is prevalent across health sciences research, and it is used to identify priorities, reach consensus on issues of importance and establish clinical guidelines. Thus, as a form of expert opinion research, it can address fundamental questions present in healthcare. However, there is little guidance on how to conduct them, resulting in heterogenous Delphi studies and methodological confusion. Therefore, the purpose of this review is to introduce the use of the Delphi method, assess the application of the Delphi technique within health sciences research, discuss areas of methodological uncertainty and propose recommendations. Advantages of the use of Delphi include anonymity, controlled feedback, flexibility for the choice of statistical analysis, and the ability to gather participants from geographically diverse areas. Areas of methodological uncertainty worthy of further discussion broadly include experts and data management. For experts, the definition and number of participants remain issues of contention, while there are ongoing difficulties with expert selection and retention. For data management, there are issues with data collection, defining consensus and methods of data analysis, such as percent agreement, central tendency, measures of dispersion, and inferential statistics. Overall, the use of Delphi addresses important issues present in health sciences research, but methodological issues remain. It is likely that the aggregation of future Delphi studies will eventually pave the way for more comprehensive reporting guidelines and subsequent methodological clarity.

International multispecialty consensus on how to image, define, and grade ultrasound imaging features of first metatarsophalangeal joint osteoarthritis, a Delphi consensus study

Objective

To reach consensus concerning which ultrasound imaging features should be assessed and graded, and what ultrasound imaging procedure should be performed when examining osteoarthritic change in the first metatarsophalangeal joint.

Design

An online Delphi study was conducted over four iterative rounds with 16 expert health professionals. Items were scored from 0 to 100 (0 ​= ​not at all important; 100 ​= ​extremely important). Consensus was defined based upon an item receiving a median score of ≥70% acceptance. Items receiving median score of ≤50% were rejected. Items considered ambiguous (median score 51%-69% of acceptance) were assessed in an additional round. A final round determined the content validity of items through calculation of the content validity ratio and content validity index.

Results

Sixteen items were deemed essential, which included osteophytes graded dichotomously, cartilage damage graded continuously, synovitis and joint space narrowing graded on a semiquantitative scale. The panel deemed essential that the first metatarsophalangeal joint start in a neutral position, then move through range of motion for both dorsal and plantar scanning, orientating the probe in longitudinal and in transverse, whilst using first metatarsal head and proximal phalanx as anatomical landmarks. A supine body position was only deemed essential for a dorsal scan and a neutral foot/ankle position was only rated essential for a plantar scan. The content validity index of the 16 essential items was 0.19.

Conclusion

The consensus exercise has identified the essential components the ultrasound imaging acquisition procedure should encompass when examining first metatarsophalangeal joint osteoarthritis.

Development and validation of the regarding infection prevention and control among environmental service workers on knowledge, attitudes, practise, and experience questionnaire

Purpose

This study aimed to develop and test the validity and reliability of the Knowledge, Attitudes, Practise, and Experience regarding Infection Prevention and Control-associated Questionnaire for environmental service workers.

Design

This study was a development and validation study of a questionnaire using multiple methods, including literature review, questionnaire survey, and Delphi technique.

Methods

Phase I of the study entailed the development of items through an extensive literature review and two round Delphi process with 15 experts specialised in infection prevention and control, environmental service worker management, or scale construction to examine the content validity of the questionnaire. Phase II involved administering the questionnaire to a convenience sample of 1,176 environmental service workers from the public hospital from 13 provinces in China to evaluate its construct validity and reliability.

Findings

In the two rounds of Delphi consultation, the recovery rate were 93.75 and 100%. Moreover, the expert authority coefficient was 0.93, and the coordination coefficients of expert opinions in the first round were as follows: correlation of 0.204 and importance of 0.249 for the first-level index; correlation of 0.128 and importance of 0.142 for the secondary index. In round two, the coordination coefficients of expert opinions were as follows: correlation of 0.221 and importance of 0.221 for the first-level indicators; correlation of 0.096 and importance of 0.101 for the secondary index. The results for the index were P < 0.05 for the two rounds. The pilot survey shows the instrument was excellent content validity (S-CVI/Ave = 0.989). The overall internal consistency was excellent (Cronbach's α = 0.967). The questionnaire ultimately comprised four first-level indices (knowledge, attitudes, practise, and experience) and 49 second-level indices.

Conclusion

The Questionnaire demonstrated good reliability and validity and is effective in measuring levels of infection prevention and control-related knowledge, attitudes, practise, and experience among environmental service workers. It will provide a tool for future national investigations of the current infection prevention and control situation among environmental service workers. Future research should explore determinants of environmental service workers' knowledge, attitudes, practise, and experience and associations between infection prevention and control knowledge, attitudes, practises, and experience.

What are the top priorities of patients and clinicians for the organization of primary cardiovascular care in Quebec? A modified e-Delphi study

BACKGROUND

Cardiovascular diseases are the leading cause of death and disability worldwide. Little is known about the organizational priorities of patients and clinicians involved in primary cardiovascular care. This study aimed to identify their shared top priorities and explore on which aspects their perspectives differed.

METHODS

A three-round modified online Delphi study was carried out with patients and clinicians in seven academic primary care settings from metropolitan, suburban and small-town areas in Quebec, Canada. Patient partners experienced in the mobilization of their experiential knowledge also participated in the study. Following an "open" round, the items elicited were assessed by a combined rating and ranking approach. Items achieving an initial consensus level ≥70% were reassessed and then rank-ordered based on their final scores. Levels of consensus achieved among patients and clinicians were compared using Fisher's Exact tests.

RESULTS

Thirty panelists completed the study (9 clinic patients, 7 patient partners and 14 clinicians). Out of 41 organizational aspects generated, six top priorities were shared by patients and clinicians. These related to listening and tailoring care to each patient, provision of personalized information, rapid response in the event of a problem, keeping professional training up-to-date, and relational and informational continuity of care. Statistically significant differences were found between patients' and clinicians' perspectives regarding the importance of offering healthy lifestyle and prevention activities at the clinic (lower for patients), timely access to the treating physician (higher for patients), and effective collaboration with specialist physicians (higher for patients).

CONCLUSION

Although their views differ on some organizational aspects, patients and clinicians share a small set of top priorities for primary cardiovascular care that may be transferable to other chronic diseases. These top priorities should remain a central focus of clinical settings, alongside other primary care reform goals.

Attitudes of psychiatrists toward telepsychiatry: A policy Delphi study

Objectives

To delineate areas of consensus and disagreements among practicing psychiatrists from various levels of clinical experience, hierarchy and organizations, and to test their ability to converge toward agreement, which will enable better integration of telepsychiatry into mental health services.

Methods

To study attitudes of Israeli public health psychiatrists, we utilized a policy Delphi method, during the early stages of the COVID pandemic. In-depth interviews were conducted and analyzed, and a questionnaire was generated. The questionnaire was disseminated amongst 49 psychiatrists, in two succeeding rounds, and areas of consensus and controversies were identified.

Results

Psychiatrists showed an overall consensus regarding issues of economic and temporal advantages of telepsychiatry. However, the quality of diagnosis and treatment and the prospect of expanding the usage of telepsychiatry to normal circumstances-beyond situations of pandemic or emergency were disputed. Nonetheless, efficiency and willingness scales slightly improved during the 2nd round of the Delphi process. Prior experience with telepsychiatry had a strong impact on the attitude of psychiatrists, and those who were familiar with this practice were more favorable toward its usage in their clinic.

Conclusions

We have delineated experience as a major impact on the attitudes toward telepsychiatry and the willingness for its assimilation in clinical practice as a legitimate and trustworthy method. We have also observed that the organizational affiliation significantly affected psychiatrists' attitude, when those working at local clinics were more positive toward telepsychiatry compared with employees of governmental institutions. This might be related to experience and differences in organizational environment. Taken together, we recommend to include hands-on training of telepsychiatry in medical education curriculum during residency, as well as refresher exercises for attending practitioners.

Diagnosis of vocal cord dysfunction / inducible laryngeal obstruction-A Delphi study protocol

INTRODUCTION

Currently there is no consistent and widely accepted approach to the diagnosis of vocal cord dysfunction/inducible laryngeal obstruction (VCD/ILO). Harmonised diagnostic methods are vital to enable optimal diagnosis, advance management and enable research. We aim to obtain consensus on how expert clinicians recognise and diagnose VCD/ILO.

METHODS AND ANALYSIS

Two-round modified Delphi, with workshop validation.

ETHICS AND DISSEMINATION

Institutional Board Review was obtained from the Monash Health Human Research Ethics Committee. The dissemination plan is for presentation and publication.

REGISTRATION DETAILS

Registered at Australia and New Zealand Clinical Trials Registry ACTRN12621001520820p.

Strategic Priorities for Implementation of Father-Inclusive Practice in Mental Health Services for Children and Families: A Delphi Expert Consensus Study

The aim of this study was to investigate expert consensus on barriers and facilitators to the organizational implementation of Father-Inclusive Practice (FIP) in child and family services to establish strategic priorities for implementation. An international panel of 56 experts in child and family service provision and father inclusion were surveyed using the Delphi technique. Three online questionnaires were used to gather opinions and measure experts’ levels of agreement in regard to factors that enable or hinder the organizational implementation of FIP. Survey design, analysis and interpretation was guided by the Consolidated Framework for Implementation Research (CFIR). Consensus was achieved for 46.4% (n = 13) statements. Eight barriers and five facilitators were identified as strategic priorities to organizational implementation of FIP. The key factors were related to the following CFIR themes: leadership engagement, access to information and knowledge, implementation climate, structural characteristics, networks and communication, client needs and resources, external policies and incentives, and reflecting and evaluating. The study findings suggest that issues related to central prioritization, top-down organizational processes and external policy context should represent priority areas for implementation. Our results prioritise methods for improving FIP by highlighting the key areas of organizational practice to be addressed by tailored implementation strategies.

Patient- and parent-reported outcome measures of developmental adaptive abilities in visually impaired children: The Visual Impairment Developmental Autonomy (VIDA) scale

In the pediatric context, parents' and patients' engagement in the care process is strongly recommended and could be pursued using patient-reported outcome measures (PROMs), which therefore become useful for planning and monitoring treatments. Nevertheless, few data are available from families of children with neurodevelopmental disorders such as visual impairment (VI). The Visual Impairment Developmental Autonomy (VIDA) project aims to develop and validate a patient- and parent-reported tool to measure the most relevant aspects concerning everyday adaptive abilities in children and adolescents with visual impairment: the VIDA scale. The present paper illustrates the Delphi process of item generation engaging parents and patients and presents a protocol for the validation of this new co-designed tool in an Italian visually impaired pediatric population. Twenty-three families and five adolescents provided a list of 192 items and assessed their relevance. Items were categorized in 5 areas of adaptive abilities (i.e., table manners, clothing, personal hygiene, orientation and mobility, and socio-affectivity) and into three age ranges based on the patient's age. The final 102-item Vida Scale will be administered to a minimum of 300 visually impaired children together with measures of quality of life and child adjustment to investigate its psychometric properties.

A scoping review of the evaluation and effectiveness of technical assistance

Background

Although the benefits of evidence-based practices (EBPs) for advancing community outcomes are well-recognized, challenges with the uptake of EBPs are considerable. Technical assistance (TA) is a core capacity building strategy that has been widely used to support EBP implementation and other community development and improvement efforts. Yet despite growing reliance on TA, no reviews have systematically examined the evaluation of TA across varying implementation contexts and capacity building aims. This study draws on two decades of peer-reviewed publications to summarize the evidence on the evaluation and effectiveness of TA.

Methods

Guided by Arksey and O’Malley’s six-stage methodological framework, we used a scoping review methodology to map research on TA evaluation. We included peer-reviewed articles published in English between 2000 and 2020. Our search involved five databases: Business Source Complete, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PsycInfo, and PubMed.

Results

A total of 125 evaluation research studies met the study criteria. Findings indicate that publications have increased over the last two decades, signaling a growth in the recognition and reporting of TA. Technical assistance is being implemented across diverse settings, often serving socially vulnerable and under-resourced populations. Most evaluation research studies involved summative evaluations, with TA outcomes mostly reported at the organizational level. Only 5% of the studies examined sustainability of TA outcomes. This review also demonstrates that there is a lack of consistent standards regarding the definition of TA and the level of reporting across relevant TA evaluation categories (e.g., cadence of contact, and directionality).

Conclusions

Advances in the science and practice of TA hinge on understanding what aspects of TA are effective and when, how, and for whom these aspects of TA are effective. Addressing these core questions requires (i) a standard definition for TA; (ii) more robust and rigorous evaluation research designs that involve comparison groups and assessment of direct, indirect, and longitudinal outcomes; (iii) increased use of reliable and objective TA measures; and (iv) development of reporting standards. We view this scoping review as a foundation for improving the state of the science and practice of evaluating TA.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00314-1.

Un-Meetings as tools for translational idea generation: A semantic content analysis of an Opioid Crisis Un-Meeting

Background

Team development and idea generation are key intertwined steps in translational science that need a framework to accommodate unstructured, participatory interactions. To this end, we introduced Un-Meetings to the Clinical and Translational Science Awards (CTSA) Program, innovative events that facilitate cross-disciplinary idea generation and informal discussions between translational scientists, policy makers, community members, advocates, and public health professionals. Here we describe a mixed methods study to characterize the conceptual diversity and clusterization of ideas generated through an Opioid Crisis Un-Meeting.

Methods

An Un-Meeting targeting translation science approaches to the opioid crisis were hosted at the University of Rochester Center for Leading Innovation and Collaboration (CLIC). We used semantic analysis and conceptual mapping of keywords to analyze how attendee-led idea generation sessions identified topics for breakout discussions.

Results

One hundred and two individuals from 40 institutions proposed 150 unique ideas that were grouped into 23 breakout sessions. Network analysis showed that diverse pools of experts were bridged by topics addressing the complexities of the opioid crisis. Two clusters emerged: (1) systems, contexts, and community engagement, and (2) technologies, innovations, and treatment advancements.

Conclusions

The cross-disciplinary nature of topic areas that bridge across thematic communities provide opportunities for CTSA programs to engage and support development of diverse translational teams. Potential opportunities for team building include technological advancements of opioid prevention, treatment, surveillance, systems approaches, and studies focusing on special populations and health disparities. The analysis method here may be useful in identifying naturally emerging teams of experts and community gaps when addressing large problems.

Developing a Health Literacy Scale for adults in Hong Kong: A modified e-Delphi study with healthcare consumers and providers

INTRODUCTION

Health literacy (HL) refers to individuals' abilities to process and use health information to promote health. This study aimed to develop the first HL measurement tool for the Chinese Hong Kong population.

METHODS

A two-phase methodology was adopted. In Phase I, evidence synthesis with a deductive method was conducted to formulate the item list from the literature. In Phase II, a modified e-Delphi survey was conducted among stakeholders (i.e., healthcare providers and healthcare consumers) to confirm the content validity of the item list. The stakeholders were invited to rate the relevance of each draft item on a 4-point scale and provide suggestions for revisions, removal or adding new items.

RESULTS

In Phase I, a total of 34 items covering functional, interactive and critical HL were generated. In Phase II, to obtain a balanced view from experts and laypeople, healthcare professionals (n = 12) and consumers (n = 12) were invited to participate in the Delphi panel. The response rates of the three rounds were 100%. After the third round, the consensus was reached for 31 items, and no further comments for adding or revising items were received. All items exhibited excellent content validity (item content validity index: 0.79-1.00; K*: 0.74-1.00).

CONCLUSIONS

A Health Literacy Scale for Hong Kong was developed. Compared with existing HL scales, the scale fully operationalized the skills involved in functional, interactive and critical HL. The Delphi study shows evidence supporting the high content validity of all items in the scale. In future studies, these items should undergo rigorous testing to examine their psychometric properties in our target population groups. By illuminating the details in the development process, this paper provides a deeper understanding of the scale's scope and limitations for others who are interested in using this tool.

PATIENT OR PUBLIC CONTRIBUTION

Public as healthcare consumers, in addition to healthcare providers, were involved in developing a new HL scale for this study. The input from the public contributed to examining the scale's content validity by judging whether all items reflected the skills that they need to find and use health-related information in their daily life.

Evaluating quality management of road construction projects: a Delphi study

Purpose

There are various methods to assess the failure of a road construction project that does not meet quality and performance standards. This study aims to evaluate the factors affecting the quality management of road construction projects in Indonesia by identifying the critical challenges in implementing quality management processes and examining the countermeasures to address these problems.

Design/methodology/approach

In this research, the quality management processes in Indonesian road construction projects are evaluated using a mixed quantitative and qualitative approach. Quantitative data obtained from two rounds of Delphi questionnaire surveys as the primary data source, and focus group interviews as the qualitative data are used to achieve the aims of this study.

Findings

It is found that from as early as in the preconstruction stage of a project, the availability of quality standards and documentation is problematic when implementing quality management processes. The critical factor of the quality control activities of a project and the quality management roles and responsibilities of the stakeholders during road construction is the project team. This constraint deals with the interaction of the competency, commitment, and mutual cooperation among the main stakeholders to ensure that quality management processes are implemented. This implementation is a part of controlling the road pavement materials, supervising the project activities according to a specified standard, and reporting the project performance. The findings of this study add parameters relating to quality management and quality processes in a road construction project area.

Originality/value

This study is the first to evaluate the quality management processes in the construction of Indonesian road projects. The investigation identifies and evaluates the causes of the persistence of poor road quality. All revealed constraints are substantial factors that hinder the implementation of quality management processes when delivering quality road products. This study addresses the key influencing factors and scenarios related to quality management during road construction projects in Indonesia and other developing countries as practical examples and provides case-based insights for construction practitioners and civil engineering academics in developed countries.

National Delphi Survey on Anesthesiology Resident Training in Perioperative Ultrasound

OBJECTIVE

To establish agreement among nationwide experts through a Delphi process on the key components of perioperative ultrasound and the recommended minimum number of examinations that should be performed by a resident upon graduation.

DESIGN

A prospective cross-sectional study.

SETTING

A survey on multiinstitutional academic medical centers.

PARTICIPANTS

Anesthesiology residency program directors and/or experts in perioperative ultrasound.

INTERVENTIONS

A list of components and examinations recommended for anesthesiology resident training in perioperative ultrasound was developed based on guidelines and 2 survey rounds among a steering committee of 10 experts. A questionnaire asking for a rating of each component on a 5-point Likert scale subsequently was sent to an expert panel of 120 anesthesiology residency program directors across the United States. An agreement of at least 70% of participants, rating a component as 4 or 5, was compulsory to list a component as essential for anesthesiology resident training in perioperative ultrasound.

MEASUREMENTS AND MAIN RESULTS

The nationwide survey's response rate was 62.5%, and agreement was reached after 2 Delphi rounds. The final list included 44 essential components for basic ultrasound physics and knobology, cardiac ultrasound, lung ultrasound, and ultrasound-guided vascular access. Agreement was not reached for abdominal ultrasound, gastric ultrasound, and ultrasound-guided airway assessment. Agreement for the recommended minimum number of examinations that should be performed by a resident upon graduation included 50 each for transthoracic and transesophageal echocardiography, and 20 each for lung ultrasound, ultrasound-guided central line, and ultrasound-guided arterial line placements.

CONCLUSIONS

The recommendations outlined in this survey can be used to establish standardized training for perioperative ultrasound by anesthesiology residency programs.

Evidence-based classification in golf for athletes with a vision impairment: A Delphi study

Vision-impaired (VI) golf is a global para-sport currently played under several different classification systems under different bodies. This study aimed to gather expert opinion to determine whether the current classification systems are fit for the purpose intended and to identify any particular issues where VI impacts the game of golf for the disabled (G4D). A panel of 20 participants with expertise in G4D took part in a three-round Delphi study. The panel agreed that the current classification system(s) for VI golf did not or only partially fulfilled the aim to minimise the impact of VI on the outcome of competition and that there should be one, internationally recognised, classification system. It was agreed that other metrics of VI, in addition to the measurement of visual acuity (VA), need to be considered. Intentional misrepresentation of VI was identified as a cause for concern. The panel agreed that the current classification system does not fully achieve its purpose. Any changes that are made to these classification systems need to be evidence based specific to VI golf. Further research is required to determine how measures of VI affect golfing performance and whether other metrics other than VA are required.

Imaging standardisation in metastatic colorectal cancer: A joint EORTC-ESOI-ESGAR expert consensus recommendation

BACKGROUND

Treatment monitoring in metastatic colorectal cancer (mCRC) relies on imaging to evaluate the tumour burden. Response Evaluation Criteria in Solid Tumors provide a framework on reporting and interpretation of imaging findings yet offer no guidance on a standardised imaging protocol tailored to patients with mCRC. Imaging protocol heterogeneity remains a challenge for the reproducibility of conventional imaging end-points and is an obstacle for research on novel imaging end-points.

PATIENTS AND METHODS

Acknowledging the recently highlighted potential of radiomics and artificial intelligence tools as decision support for patient care in mCRC, a multidisciplinary, international and expert panel of imaging specialists was formed to find consensus on mCRC imaging protocols using the Delphi method.

RESULTS

Under the guidance of the European Organisation for Research and Treatment of Cancer (EORTC) Imaging and Gastrointestinal Tract Cancer Groups, the European Society of Oncologic Imaging (ESOI) and the European Society of Gastrointestinal and Abdominal Radiology (ESGAR), the EORTC-ESOI-ESGAR core imaging protocol was identified.

CONCLUSION

This consensus protocol attempts to promote standardisation and to diminish variations in patient preparation, scan acquisition and scan reconstruction. We anticipate that this standardisation will increase reproducibility of radiomics and artificial intelligence studies and serve as a catalyst for future research on imaging end-points. For ongoing and future mCRC trials, we encourage principal investigators to support the dissemination of these imaging standards across recruiting centres.

Feasibility of Parkour-style training in team sport practice: A Delphi study

Research has suggested Parkour-style training could act as a donor sport for athlete development in team sports. This study aimed to interrogate expert consensus on the feasibility of integrating Parkour-style training into team sport practice, by employing a three-round, online Delphi method. Talent development and strength and conditioning coaches working in team sport settings were invited to participate. Twenty-four coaches completed Round One, 21 completed Round Two and 20 completed Round Three. In Round One, coaches answered 15 open-ended questions across four categories: (1) General Perceptions of Parkour-style training; (2) Potential Applications of Parkour-style training; (3) Designing and Implementing Parkour-style training Environments; and (4), Creating an Inclusive Learning Environment. Responses from Round One were analysed using reflexive thematic analysis with deductive and inductive coding resulting in 78 statements across three dimensions (Application of Parkour Style Training in Team Sports; Designing and Implementing Parkour-style training Environments; Overcoming Potential Barriers when Integrating Parkour-style training). In Rounds Two and Three, coaches rated these statements using a four-point Likert scale and measures of collective agreement or disagreement were calculated. This study established consensus around a set of design principles for integrating Parkour-style training into team sport practice routines.

Consensus on how to optimise patient/family engagement in hospital planning and improvement: a Delphi survey

OBJECTIVE

Patient and family engagement (PE) in health service planning and improvement is widely advocated, yet little prior research offered guidance on how to optimise PE, particularly in hospitals. This study aimed to engage stakeholders in generating evidence-informed consensus on recommendations to optimise PE.

DESIGN

We transformed PE processes and resources from prior research into recommendations that populated an online Delphi survey.

SETTING AND PARTICIPANTS

Panellists included 58 persons with PE experience including: 22 patient/family advisors and 36 others (PE managers, clinicians, executives and researchers) in round 1 (100%) and 55 in round 2 (95%).

OUTCOME MEASURES

Ratings of importance on a seven-point Likert scale of 48 strategies organised in domains: engagement approaches, strategies to integrate diverse perspectives, facilitators, strategies to champion engagement and hospital capacity for engagement.

RESULTS

Of 50 recommendations, 80% or more of panellists prioritised 32 recommendations (27 in round 1, 5 in round 2) across 5 domains: 5 engagement approaches, 4 strategies to identify and integrate diverse patient/family advisor perspectives, 9 strategies to enable meaningful engagement, 9 strategies by which hospitals can champion PE and 5 elements of hospital capacity considered essential for supporting PE. There was high congruence in rating between patient/family advisors and healthcare professionals for all but six recommendations that were highly rated by patient/family advisors but not by others: capturing diverse perspectives, including a critical volume of advisors on committees/teams, prospectively monitoring PE, advocating for government funding of PE, including PE in healthcare worker job descriptions and sharing PE strategies across hospitals.

CONCLUSIONS

Decision-makers (eg, health system policy-makers, hospitals executives and managers) can use these recommendations as a framework by which to plan and operationalise PE, or evaluate and improve PE in their own settings. Ongoing research is needed to monitor the uptake and impact of these recommendations on PE policy and practice.

Existing guidance on reporting of consensus methodology: a systematic review to inform ACCORD guideline development

OBJECTIVE

To identify evidence on the reporting quality of consensus methodology and to select potential checklist items for the ACcurate COnsensus Reporting Document (ACCORD) project to develop a consensus reporting guideline.

DESIGN

Systematic review.

DATA SOURCES

Embase, MEDLINE, Web of Science, PubMed, Cochrane Library, Emcare, Academic Search Premier and PsycINFO from inception until 7 January 2022.

ELIGIBILITY CRITERIA

Studies, reviews and published guidance addressing the reporting quality of consensus methodology for improvement of health outcomes in biomedicine or clinical practice. Reports of studies using or describing consensus methods but not commenting on their reporting quality were excluded. No language restrictions were applied.

DATA EXTRACTION AND SYNTHESIS

Screening and data extraction of eligible studies were carried out independently by two authors. Reporting quality items addressed by the studies were synthesised narratively.

RESULTS

Eighteen studies were included: five systematic reviews, four narrative reviews, three research papers, three conference abstracts, two research guidance papers and one protocol. The majority of studies indicated that the quality of reporting of consensus methodology could be improved. Commonly addressed items were: consensus panel composition; definition of consensus and the threshold for achieving consensus. Items least addressed were: public patient involvement (PPI); the role of the steering committee, chair, cochair; conflict of interest of panellists and funding. Data extracted from included studies revealed additional items that were not captured in the data extraction form such as justification of deviation from the protocol or incentives to encourage panellist response.

CONCLUSION

The results of this systematic review confirmed the need for a reporting checklist for consensus methodology and provided a range of potential checklist items to report. The next step in the ACCORD project builds on this systematic review and focuses on reaching consensus on these items to develop the reporting guideline.

PROTOCOL REGISTRATION

https://osf.io/2rzm9.

Informing the Design of Data Visualization Tools to Monitor the COVID-19 Pandemic in Portugal: A Web-Delphi Participatory Approach

Due to the large amount of data generated by new technologies and information systems in the health arena, health dashboards have become increasingly popular as data visualization tools which stimulate visual perception capabilities. Although the importance of involving users is recognized in dashboard design, a limited number of studies have combined participatory methods with visualization options. This study proposes a novel approach to inform the design of data visualization tools in the COVID-19 context. With the objective of understanding which visualization formats should be incorporated within dashboards for the COVID-19 pandemic, a specifically designed Web-Delphi process was developed to understand the preferences and views of the public in general regarding distinct data visualization formats. The design of the Delphi process aimed at considering not only the theory-based evidence regarding input data and visualization formats but also the perception of final users. The developed approach was implemented to select appropriate data visualization formats to present information commonly used in public web-based COVID-19 dashboards. Forty-seven individuals completed a two-round Web-Delphi process that was launched through a snowball approach. Most respondents were young and highly educated and expressed to prefer distinct visualisation formats for different types of indicators. The preferred visualization formats from the participants were used to build a redesigned version of the official DGS COVID-19 dashboard used in Portugal. This study provides insights into data visualization selection literature, as well as shows how a Delphi process can be implemented to assist the design of public health dashboards.

Hemşireler Pandemilerde Neler Yaşıyor? Bir Ölçek Geliştirme Çalışması

Aim: To develop a valid and reliable measurement tool to identify the challenges nurses face during pandemics.Method: The study has a cross-sectional and methodological design. Using a multi-method approach, it was conducted with 405 nurses in Istanbul between May 15 and August 15, 2020. Data were collected online via the Personal Information Form and Draft Scale. Descriptive statistics, validity reliability analysis, and total score averages were evaluated for data analysis using SPSS 21 and AMOS 22 programs.Results: 38 items were grouped into four dimensions. The variance rate was found as 66.88%. Goodness-of-fit statistics were appropriate. The total correlation scores of the items were between 0.42-0.82 (p&lt;0.001). Cronbach's alpha value was 0.92. There was no significant difference between the test-retest mean scores (t:1.349, p:0.188), while a highly significant correlation was found between the measurements (r:0.88 p&lt;0.001). The overall mean score of the scale was 3.50±0.62.Conclusions: The valid and reliable scale can assist nurses, managers of healthcare institutions, and policymakers in developing coping strategies for the challenges.