Experts' perceptions on motivators and barriers of healthy and sustainable dietary behaviors among adolescents: The SWITCH project

The interplay of influences shaping dietary behaviors of adolescents needs to be well-understood to develop effective strategies stimulating healthy and sustainable behaviors. This study aimed to identify the most relevant, urgent, changeable and effective motivators and barriers of healthy and sustainable dietary behaviors among adolescents (10-19 years), as perceived by an inter- and transdisciplinary expert panel. Experts working in practice (N = 19) and academia (N = 13) - in the field of sustainability, health, nutrition and/or education - participated in this exploratory mixed-methods study. Five online semi-structured focus groups were conducted (6-7 participants). Data was thematically analyzed by two coders independently, using the socioecological framework. Subsequently, the same experts individually selected the five most relevant determinants and rated those on their urgency, changeability and effectiveness through an online questionnaire (N = 21). Participants revealed a wide system of 31 main determinants including 173 sub-determinants that motivate or hinder healthy and sustainable eating among adolescents. This system of determinants was mapped on the different layers of the socioecological model: individual factors (25 motivators; 30 barriers), social environments (15 motivators; 13 barriers), physical environments (18 motivators; 15 barriers), macro-level environments (19 motivators; 38 barriers). 'Role of peers' (social environment) was selected most as a determinant to be targeted in interventions (N = 13; urgency (M = 6.38) changeability (M = 3.85), effectiveness (M = 5.62)), followed by 'food environment around school' (N = 9; urgency (M = 5.78) changeability (M = 3.44), effectiveness (M = 5.44)), 'social influences' (N = 7; urgency (M = 5.43) changeability (M = 4.00), effectiveness (M = 5.71)), 'autonomy in development' (N = 7; urgency (M = 6.00) changeability (M = 4.29), effectiveness (M = 5.86)) and 'food availability' (N = 7; urgency (M = 6.29) changeability (M = 3.29), effectiveness (M = 6.29)). The prioritized determinants indicate that adolescents should be provided more supportive social and physical environments that promote healthy and sustainable dietary behaviors, taking into account their growing autonomy.

Derivation of no significant risk levels for three lower acrylates: Conclusions and recommendations from an expert panel

A panel of toxicology, mode of action (MOA), and cancer risk assessment experts was engaged to derive no-significant-risk-levels (NSRLs) for three lower acrylates: methyl acrylate (MA), ethyl acrylate (EA), and 2-ethylhexyl acrylate (2EHA) using the best available science, data, and methods. The review was structured as a five-round, modified Delphi format, a systematic process for collecting independent and deliberative input from panel members, and it included several procedural elements to reduce potential sources of bias and groupthink. Input from the panel for key decisions in the dose-response assessments resulted in NSRL values of 530 μg/day (330-800 μg/day), 640 μg/day (280-670 μg/day), and 1700 μg/day (1300-2700 μg/day) for MA, EA, and 2EHA, respectively. Novel to this approach were the use of nonneoplastic lesions reported at point of contact where tumors have been reported in laboratory rodents, along with nonlinear extrapolation to low doses (uncertainty factor approach) based upon panel recommendations. Confidence in these values is considered medium to high for exposures applied to the routes of exposure tested (inhalation for MA and EA, dermal for 2EHA), but confidence is considered lower when applied to other routes of exposure.

Priorities for implementation research on diagnosing cancer in primary care: a consensus process

BACKGROUND

The early detection and diagnosis of cancer to reduce avoidable mortality and morbidity is a challenging task in primary health care. There is a growing evidence base on how to enable earlier cancer diagnosis, but well-recognised gaps and delays exist around the translation of new research findings into routine clinical practice. Implementation research aims to accelerate the uptake of evidence by health care systems and professionals. We aimed to identify priorities for implementation research in early cancer diagnosis in primary care.

METHODS

We used a RAND/UCLA modified Delphi consensus process to identify and rank research priorities. We asked primary care physicians, patients and researchers to complete an online survey suggesting priorities for implementation research in cancer detection and diagnosis. We summarised and presented these suggestions to an 11-member consensus panel comprising nine primary care physicians and two patients. Panellists independently rated the importance of suggestions on a 1-9 scale (9 = very high priority; 1 = very low priority) before and after a structured group discussion. We ranked suggestions using median ratings.

RESULTS

We received a total of 115 suggested priorities for implementation research from 32 survey respondents (including 16 primary care professionals, 11 researchers, and 4 patient and public representatives; 88% of respondents were UK-based). After removing duplicates and ineligible suggestions, we presented 37 suggestions grouped within 17 categories to the consensus panel. Following two rounds of rating, 27 suggestions were highly supported (median rating 7-9). The most highly rated suggestions concerned diagnostic support (e.g., access to imaging) interventions (e.g., professional or patient education), organisation of the delivery of care (e.g., communication within and between teams) and understanding variations in care and outcomes.

CONCLUSIONS

We have identified a set of priorities for implementation research on the early diagnosis of cancer, ranked in importance by primary care physicians and patients. We suggest that researchers and research funders consider these in directing further efforts and resources to improve population outcomes.

Caring for high-need patients

OBJECTIVE

We aimed to explore the construct of "high need" and identify common need domains among high-need patients, their care professionals, and healthcare organizations; and to describe the interventions that health care systems use to address these needs, including exploring the potential unintended consequences of interventions.

METHODS

We conducted a modified Delphi panel informed by an environmental scan. Expert stakeholders included patients, interdisciplinary healthcare practitioners (physicians, social workers, peer navigators), implementation scientists, and policy makers. The environmental scan used a rapid literature review and semi-structured interviews with key informants who provide healthcare for high-need patients. We convened a day-long virtual panel meeting, preceded and followed by online surveys to establish consensus.

RESULTS

The environmental scan identified 46 systematic reviews on high-need patients, 19 empirical studies documenting needs, 14 intervention taxonomies, and 9 studies providing construct validity for the concept "high need." Panelists explored the construct and terminology and established that individual patients' needs are unique, but areas of commonality exist across all high-need patients. Panelists agreed on 11 domains describing patient (e.g., social circumstances), 5 care professional (e.g., communication), and 8 organizational (e.g., staffing arrangements) needs. Panelists developed a taxonomy of interventions with 15 categories (e.g., care navigation, care coordination, identification and monitoring) directed at patients, care professionals, or the organization. The project identified potentially unintended consequences of interventions for high-need patients, including high costs incurred for patients, increased time and effort for care professionals, and identification of needs without resources to respond appropriately.

CONCLUSIONS

Care for high-need patients requires a thoughtful approach; differentiating need domains provides multiple entry points for interventions directed at patients, care professionals, and organizations. Implementation efforts should consider outlined intended and unintended downstream effects on patients, care professionals, and organizations.

Quantification of pharmaceuticals in hospital effluent: Weighted ranking of environmental risk using a fuzzy hybrid multicriteria method

An analytical method for quantification of seventeen pharmaceuticals and one metabolite was validated and applied in the analysis of hospital effluent samples. Two different sampling strategies were used: seasonal sampling, with 7 samples collected bimonthly; and hourly sampling, with 12 samples collected during 12 h. Thus, the variability was both seasonal and within the same day. High variability was observed in the measured concentrations of the pharmaceuticals and the metabolite. The quantification method, performed using weighted linear regression model, demonstrated results of average concentrations in seasonal samples ranged between 0.19 μgL-1 (carbamazepine) and higher than 61.56 μgL-1 (acetaminophen), while the hourly samples showed average concentrations between 0.07 μgL-1 (diazepam) and higher than 54.91 μgL-1 (acetaminophen). It is described as higher because the maximum concentration of the calibration curve took into account the dilution factor provided by DLLME. The diurnal results showed a trend towards higher concentrations in the first and last hours of sampling. The risk quotient (RQ) was calculated using organisms from three different trophic levels, for all the analytes quantified in the samples. Additionally, in order to understand the level of importance of each RQ, an expert panel was established, with contributions from 23 specialists in the area. The results were analyzed using a hybrid decision-making approach based on a Fuzzy Analytic Hierarchy Process (FAHP) and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method, in order to rank the compounds by environmental risk priority. The compounds of greatest concern were losartan, acetaminophen, 4-aminoantipyrine, sulfamethoxazole, and metoclopramide. Comparison of the environmental risk priority ranking with the potential human health risk was performed by applying the same multicriteria approach, with the prediction of endpoints using in silico (Q)SAR models. The results obtained suggested that sulfamethoxazole and acetaminophen were the most important analytes to be considered for monitoring.

Cancer weight of evidence for three lower acrylates: Conclusions and recommendations from an expert panel

An international panel of experts was engaged to assess the cancer weight of evidence (WOE) for three lower acrylates: methyl acrylate, ethyl acrylate, and 2-ethylhexyl acrylate. The review was structured as a three-round, modified Delphi format, a systematic process for collecting independent and deliberative input from panel members, and it included procedural elements to reduce bias and groupthink. Based upon the available science, the panel concluded: (1) The MOA for point of contact tumors observed in rodent cancer bioassays that is best supported by available data involves increased cell replication by cytotoxicity and regenerative proliferation; (2) The WOE supports a cancer classification of "Not likely to be carcinogenic to humans" a conclusion that is more in line with an IARC classification of Group 3 rather than Group 2 B; (3) Quantitative cancer potency values based on rodent tumor data are not required for these chemicals; and (4) Human health risk assessment for these chemicals should instead rely on non-cancer, precursor endpoints observed at the point of contact (e.g., hyperplasia). The degree of consensus (consensus scores of 0.84-0.91 out of a maximum score of 1) and degree of confidence (7.7-8.7 out of a maximum score of 10) in the WOE conclusions is considered high.

Research priorities for therapeutic plasma exchange in critically ill patients

Therapeutic plasma exchange (TPE) is a therapeutic intervention that separates plasma from blood cells to remove pathological factors or to replenish deficient factors. The use of TPE is increasing over the last decades. However, despite a good theoretical rationale and biological plausibility for TPE as a therapy for numerous diseases or syndromes associated with critical illness, TPE in the intensive care unit (ICU) setting has not been studied extensively. A group of eighteen experts around the globe from different clinical backgrounds used a modified Delphi method to phrase key research questions related to "TPE in the critically ill patient". These questions focused on: (1) the pathophysiological role of the removal and replacement process, (2) optimal timing of treatment, (3) dosing and treatment regimes, (4) risk-benefit assumptions and (5) novel indications in need of exploration. For all five topics, the current understanding as well as gaps in knowledge and future directions were assessed. The content should stimulate future research in the field and novel clinical applications.

Formative evaluation of a STEAM and nutrition education summer program for low-income youth

In-depth formative evaluations are vital for curriculum development and program planning but are often not conducted before a program pilots. A formative evaluation of Project stRIde was conducted to gain insight from experts and identify revisions to the curriculum. Project stRIde is a science, technology, engineering, arts, and mathematics (STEAM) and nutrition-based curriculum developed for 4th and 5th grade students from low-income and diverse families. Nine experts spanning the fields of nutrition education, cultural competency, elementary education, summer programs, and STEAM outreach were recruited to participate in an expert content review (ECR) survey and virtual interviews. Seven core themes were identified: effectively promoting student engagement, increased guidance or support needed, activity too difficult for age, time, confidence in teaching lessons, cultural appropriateness, and strengths of curriculum in promoting STEAM education and innovation. Across the lessons, all reviewers agreed that the lessons were accurate, incorporated STEAM concepts, and were culturally appropriate for this population. Future major edits to the curriculum include creating supplemental videos, modifying some activities for age level, and incorporating more opportunities for participant engagement. Overall, an ECR is an effective way to examine a program's strengths and limitations and should be included in the beginning stages of program planning.

An evaluation of reproductive toxicity studies and data interpretation of N-methylpyrrolidone for risk assessment: An expert panel review

An expert panel was assembled to evaluate reproductive toxicology study data and their application to health risk assessment to provide input on the data quality, interpretation, and application of data from three multi-generation reproductive toxicity studies of N-methylpyrrolidone (NMP). Panelists were engaged using a double-blinded, modified Delphi format that consisted of three rounds. Key studies were scored using the U.S. Environmental Protection Agency's (EPA) questions and general considerations to guide the evaluation of experimental animal studies for systematic review. The primary conclusions of the panel are that one of the studies (Exxon, 1991) is not a high-quality study due to several design flaws that includes: (1) exceedance of the maximum tolerable dose in the high dose group; (2) failure to adjust feed concentrations of NMP during the lactation period, resulting in NMP doses that were 2- to 3-fold higher than nominal levels; and/or (3) underlying reproductive performance problems in the strain of rats used. For these reasons, the panel recommended that this study should not be considered for quantitative risk assessment of NMP. Exclusion of this study, and its corresponding data for male fertility and female fecundity, from the quantitative risk assessment results in a change in the identification of the most sensitive endpoint. Instead, changes in rat fetal/pup body weight, an endpoint previously selected by EPA, was identified as an appropriate basis for human health risk assessment based on a consideration of the best available science and weight of scientific evidence supported by the NMP toxicity database.

A novel method for the translation and cross-cultural adaptation of health-related quality of life patient-reported outcome measurements

BACKGROUND

This paper presents a novel methodology for translation and cross-cultural adaptation of health-related quality-of-life patient-reported outcome measures, incorporating the Delphi method. Specifically, we describe the process of translating the Pelvic Floor Distress Inventory-20 and Pelvic Floor Impact Questionnaire-7 from English to Norwegian using this method.

METHODS

The multistep translation method combined the European Organization for Research and Treatment of Cancer Quality of Life guidelines, an Expert Panel review, and the Delphi method. It comprised two independent forward- and back-translations. While the bilingual pelvic floor Expert Panel ensured rigorous cross-checking and effective cross-cultural adaptation, the addition of the Delphi method (comprising the attributes of anonymity, controlled feedback, and statistical group response) further established consensus on translated items.

OUTCOMES

The application of the Delphi method in the Expert Panel phase proved adequate in producing comprehensible intermediate Norwegian versions ready for pilot testing. The Expert Panel reviewed the comments made by patients completing the instruments and offered advice to allow final translated versions to be produced and tested for measurement properties. This iterative approach, internal logic, and anonymity between rounds improved the evaluations that the panel members provided, which in turn enhanced the final translated Patient Reported Outcome Measures (PROMs).

CONCLUSIONS

To our knowledge, this work represents the first demonstration of the application of an Expert Panel review incorporating a Delphi method to assess health-related quality-of-life instruments. The controlled feedback approach, iterative nature, internal logic, and anonymity of the Delphi consensus method appeared to ensure a good cross-cultural adaptation of these PROMs.

Challenges and shortcomings of antibacterial discovery projects

OBJECTIVES

Antibacterial drug discovery activities are essential for filling clinical pipelines with promising clinical candidates. Little information is available about the challenges and shortcomings of small companies and academic institutions in performing these important discovery tasks.

METHODS

We performed a content analysis of 463 reviewer comments on 91 funding applications of antibacterial drug discovery projects submitted to two major global funders between 2016 and 2020 that had not proceeded further in the selection process. This quality assessment was complemented with the inputs (via e-mail) from a panel involving six antibiotic research and development (R&D) experts with long-standing expertise and experience in antibiotic drug discovery.

RESULTS

Common critical comments of reviewers are grouped into three main categories: scientific and technical shortcomings, unclear potential societal impact, and insufficient capability and expertise of the project team regarding the R&D process. Insufficient characterization of in vitro activity and/or testing of the hits/leads and insufficient antibacterial activity were the most common critical comments. Other areas of concern were insufficient or lack of differentiation from available drugs or projects with a long R&D history, and the research team's insufficient knowledge of a structured streamlined R&D process as reflected in severe gaps in the expertise of the R&D team. Little appreciation for the problem of the emergence of target-based resistance, especially in single-target approaches, and little awareness of toxicological issues, including approaches with historical liabilities were also commonly mentioned. The shortcomings identified through the analysis of funding applications are echoed by the results of the expert panel.

DISCUSSION

Our analysis identified an urgent need of strengthening the support for antibacterial drug discovery teams to help more projects reach such a quality to be eligible for global funders and private investors.

Towards health-based nano reference values (HNRVs) for occupational exposure: Recommendations from an expert panel

Unique physicochemical characteristics of engineered nanomaterials (ENMs) suggest the need for nanomaterial-specific occupational exposure limits (OELs). Setting these limits remains a challenge. Therefore, the aim of this study was to set out a framework to evaluate the feasibility of deriving advisory health-based occupational limit values for groups of ENMs, based on scientific knowledge. We have used an expert panel approach to address three questions: 1) What ENM-categories should be distinguished to derive advisory health-based occupational limit values (or health-based Nano Reference Values, HNRVs) for groups of ENMs? 2) What evidence would be needed to define values for these categories? And 3) How much effort would it take to achieve this? The panel experts distinguished six possible categories of HNRVs: A) WHO-fiber-like high aspect ratio ENMs (HARNs), B) Non-WHO-fiber-like HARNs and other non-spheroidal ENMs, C) readily soluble spheroidal ENMs, D) biopersistent spheroidal ENMs with unknown toxicity, E) biopersistent spheroidal ENMs with substance-specific toxicity and F) biopersistent spheroidal ENMs with relatively low substance-specific toxicity. For category A, the WHO-fiber-like HARNs, agreement was reached on criteria defining this category and the approach of using health-based risk estimates for asbestos to derive the HNRV. For category B, a quite heterogeneous category, more toxicity data are needed to set an HNRV. For category C, readily soluble spheroidal ENMs, using the OEL of their molecular or ionic counterpart would be a good starting point. For the biopersistent ENMs with unknown toxicity, HNRVs cannot be applied as case-by-case testing is required. For the other biopersistent ENMs in category E and F, we make several recommendations that can facilitate the derivation of these HNRVs. The proposed categories and recommendations as outlined by this expert panel can serve as a reference point for derivation of HNRVs when health-based OELs for ENMs are not yet available.

Expert opinion on the recognition, diagnosis and management of children and adults with Fabry disease: a multidisciplinary Turkey perspective

This consensus statement by a panel of Fabry experts aimed to identify areas of consensus on conceptual, clinical and therapeutic aspects of Fabry disease (FD) and to provide guidance to healthcare providers on best practice in the management of pediatric and adult patients with FD. This consensus statement indicated the clinical heterogeneity of FD as well as a large number of pathogenic variants in the GLA gene, emphasizing a need for an individualized approach to patient care. The experts reached consensus on the critical role of a high index of suspicion in symptomatic patients and screening of certain at-risk groups to reveal timely and accurate diagnosis of FD along with an increased awareness of the treating physician about the different kinds of pathogenic variants and their clinical implications. The experts emphasized the crucial role of timely recognition of FD with minimal delay from symptom onset to definite diagnosis in better management of FD patients, given the likelihood of changing the disease's natural history, improving the patients' quality of life and the prognosis after enzyme replacement therapy (ERT) administered through a coordinated, multidisciplinary care approach. In this regard, this consensus document is expected to increase awareness among physicians about unique characteristics of FD to assist clinicians in recognizing FD with a well-established clinical suspicion consistent with pathogenic variants and gender-based heterogeneous clinical manifestations of FD and in translating this information into their clinical practice for best practice in the management of patients with FD.

Prioritized research recommendations and potential solutions: addressing gaps surrounding dietary supplement ingredients for boosting brain health and optimizing cognitive performance

In the market of dietary supplements, a low level of certainty exists in the state of the science, coupled with not always knowing what is in the product. Together these issues make weighing benefits/risks difficult and hinder the ability to guide evidence-based practice decisions. The authors sought to identify priorities and develop potential solutions to address research gaps so that information disseminated, can ultimately, be relied upon, when trying to make appropriate and safe decisions. Using a modified-Delphi process, 8 panelists reviewed evidence, provided from systematic review, on dietary supplement ingredients for brain health, and prioritized gaps identified and offered potential solutions. Research gaps specific to dietary supplements research included the need for quality testing of products, the question of bioavailability and absorption of ingredients, and optimal composition and standardization of supplements under investigation. Other gaps related to populations studied; a general sense of bias towards focusing research on diseased rather than maintaining or optimizing performance in healthy populations. Additionally, the lack of uniform cognitive performance measures and metrics used across research is a gap, as well as whether the metrics are accurate representations of or even generalizable to "real-life" participants wishing to optimize their performance. Methodological quality and ethical concerns in the conduct and reporting of science encompass all issues. If resources map to potential solutions outlined in this paper, then these proposed next steps offered will help facilitate meaningful research, move evidence into practice recommendations, and ultimately develop better decision-making tools for consumers to trust and rely upon for making safe supplement decisions.

Development of a core evaluation framework of value-added medicines: report 1 on methodology and findings

Background

Medicines that are based on known molecules and are further developed to address healthcare needs and deliver relevant improvement for patients, healthcare professionals and/or payers are called value-added medicines (VAMs). The evaluation process of VAMs is heterogeneous across countries, and it has been primarily designed for originator pharmaceuticals with confirmatory evidence collected alongside pivotal clinical trials. There is a mismatch between evidence requirements by public decision-makers and evidence generated by manufacturers of VAMs. Our objective was to develop a core evaluation framework for VAMs.

Methods

Potential benefits offered by VAMs were collected through a systematic literature review and allocated to separate domains in an iterative process. The draft list of domains and their applicability were validated during two consecutive virtual workshops by health policy experts representing countries with different economic statuses, geographical and decision-making contexts.

Results

Based on 158 extracted studies, the final consensus on the evaluation framework resulted in 11 value domains in 5 main clusters, including unmet medical needs, health gain (measured by health care professionals), patient-reported outcomes, burden on households, and burden on the health care system.

Conclusions

The proposed framework could reduce the heterogeneity in value assessment processes across countries and create incentives for manufacturers to invest in incremental innovation. However, some domains may not be equally relevant or accepted in all countries, therefore the core framework needs thorough adaptation in specific jurisdictions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-021-00311-6.

Delphi methodology in healthcare research: How to decide its appropriateness

The Delphi technique is a systematic process of forecasting using the collective opinion of panel members. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine. The Delphi methods assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically/logistically difficult or evidence is conflicting. However, the attempts to assess the quality standard of Delphi studies have reported significant variance, and details of the process followed are usually unclear. We recommend systematic quality tools for evaluation of Delphi methodology; identification of problem area of research, selection of panel, anonymity of panelists, controlled feedback, iterative Delphi rounds, consensus criteria, analysis of consensus, closing criteria, and stability of the results. Based on these nine qualitative evaluation points, we assessed the quality of Delphi studies in the medical field related to coronavirus disease 2019. There was inconsistency in reporting vital elements of Delphi methods such as identification of panel members, defining consensus, closing criteria for rounds, and presenting the results. We propose our evaluation points for researchers, medical journal editorial boards, and reviewers to evaluate the quality of the Delphi methods in healthcare research.

Delphi methodology in healthcare research: How to decide its appropriateness

The Delphi technique is a systematic process of forecasting using the collective opinion of panel members. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine. The Delphi methods assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically/logistically difficult or evidence is conflicting. However, the attempts to assess the quality standard of Delphi studies have reported significant variance, and details of the process followed are usually unclear. We recommend systematic quality tools for evaluation of Delphi methodology; identification of problem area of research, selection of panel, anonymity of panelists, controlled feedback, iterative Delphi rounds, consensus criteria, analysis of consensus, closing criteria, and stability of the results. Based on these nine qualitative evaluation points, we assessed the quality of Delphi studies in the medical field related to coronavirus disease 2019. There was inconsistency in reporting vital elements of Delphi methods such as identification of panel members, defining consensus, closing criteria for rounds, and presenting the results. We propose our evaluation points for researchers, medical journal editorial boards, and reviewers to evaluate the quality of the Delphi methods in healthcare research.

Upgrading current multi-attribute decision-making with a 3-dimensional decision matrix for future-based decisions

Two shortcomings of existing Multi-Attribute Decision-Making (MADM) approaches are presented in this paper. The problems were decision makers' (experts') dynamic level of experience in certain areas, as well as the difficulties of forecasting the future with the existing MADM techniques. A solution is also proposed to overcome the issues.•Two critical shortcomings of existing multi-attribute decision-making (MADM) approaches are presented with this work.•As a solution for the shortcomings, a 3-dimensional decision matrix is introduced.

The Impact of Surgeon Experience on Script Concordance Test Scoring

OBJECTIVE

The Script Concordance Test (SCT) is a test of clinical decision-making that relies on an expert panel to create its scoring key. Existing literature demonstrates the value of specialty-specific experts, but the effect of experience among the expert panel is unknown. The purpose of this study was to explore the role of surgeon experience in SCT scoring.

DESIGN

An SCT was administered to 29 general surgery residents and 14 staff surgeons. Staff surgeons were stratified as either junior or senior experts based on years since completing residency training (<15 versus >25 years). The SCT was scored using the full expert panel, the senior panel, the junior panel, and a subgroup junior panel in practice <5 years. A one-way ANOVA was used to compare the scores of first (R1) and fifth (R5) year residents using each scoring scheme. Cognitive interviews were analyzed for differences between junior and senior expert panelist responses.

RESULTS

There was no statistically significant difference between the mean score of six R1s and five R5s using the full expert panel (R1 69.08 versus R5 67.06, F_1,9 = 0.10, P = 0.76), the junior panel (R1 66.73 versus R5 62.50, F_1,9 = 0.35, P = 0.57), or the subgroup panel in practice <5 years (R1 61.07 versus R5 58.79, F_1,9 = 0.18, P = 0.75). However, the average score of R1s was significantly lower than R5s when using the senior faculty panel (R1 52.04 versus R5 63.26, F_1,9 = 26.90, P = 0.001). Cognitive interview data suggests that some responses of junior experts demonstrate less confidence than those of senior experts.

CONCLUSIONS

SCT scores are significantly affected by the responses of the expert panel. Expert differences between first and fifth year residents were only demonstrated when using an expert panel consisting of senior faculty members. Confidence may play a role in the response selections of junior experts. When constructing an SCT expert panel, consideration must be given to the experience of panel members.

The initial assessment of expert panel performance in core hospitals for cancer genomic medicine in Japan

BACKGROUND

Since June 2019, cancer genomic profiling (CGP) tests have been reimbursed by the National Health Insurance system in Japan, with restrictions for government-designated hospitals with a molecular tumor board composed of multidisciplinary specialists, known as an expert panel (EP). The standardization of EPs is a critical challenge for implementing precision oncology in the clinical setting.

METHODS

Data on consecutive cases who underwent the CGP tests at 11 core hospitals between June 2019 and January 2020 were collected. We evaluated the proportions of cases that received genomically matched treatments, including investigational new drugs (INDs) based on CGP results, and/or for which genetic counseling was recommended. Two simulated cases were annotated by each EP. The annotated reports were then centrally assessed.

RESULTS

Each EP mainly discussed the applicability to genomically matched treatments and the necessity of performing genetic counseling. A pre-review of the report by key members in each EP reportedly made the EP conference more interactive and efficient, and thereby saved time. A total of 747 cases underwent CGP tests, 28 cases (3.7%) received genomically matched treatment, and 17 cases (2.3%) were referred for genetic counseling. Annotated reports for the simulated cases varied across the EPs, particularly the number of recommended IND trials, which seemed to be associated with the actual number of participants in IND trials.

CONCLUSIONS

This investigation provides reference data for the application of precision oncology in a clinical setting. Further investigations on the standardization of clinical annotations are warranted.

Sensory typicity of regional Australian Cabernet Sauvignon wines according to expert evaluations and descriptive analysis

The concept of wine typicity has been an important tool for the international wine trade, and especially for Old World wine producing countries, where provenance criteria are regulated and act as a quality indicator. Provenance in Australia is governed by Geographical Indications, for which typicity should also be evident in terms of regional sensory profiles of wine from a given grape cultivar. Two approaches were used to identify sensory drivers for regional typicity of commercial Cabernet Sauvignon wines from three Australian regions, namely Coonawarra, Margaret River, and Yarra Valley. Cabernet Sauvignon-dominant wines from Bordeaux were also assessed for benchmarking purposes. A set of 84 wines underwent a sorting task and rate-all-that-apply (RATA) analysis of the sorted groups with an expert panel. Agglomerative hierarchical clustering of the sorting task data did not show a clear regional driver upon separating the samples into four main clusters, although certain sensory traits could be associated with the different clusters. On the other hand, canonical variate analysis (CVA) of the group-RATA results indicated several sensory drivers for the separation between the regions, such as 'mint' and 'dark fruits' being important for Coonawarra wine profiles, 'floral' and 'green pepper' for Margaret River, 'stemmy' for Yarra Valley, and 'barnyard' and 'savoury' differentiating Bordeaux wines from the other regions. A subset (n = 52) of wines was selected for further evaluation by descriptive analysis with a trained panel. Statistical evaluation with CVA revealed similar results to the expert evaluation, with Bordeaux wines showing more dissimilarity when compared to Australian regions, and having 'savoury' and 'earthy' as significant characters. The results also demonstrated that 'mint' and 'Mallee leaf' were relevant characters for Coonawarra regional sensory profile, 'violets' and 'red fruits' for Margaret River, and 'cooked vegetables' for Yarra Valley. Analysing both data sets (expert RATA and DA) revealed some agreement between the sets of results for attributes such as 'mint', 'cooked vegetables', 'floral', green-related characters, and oak characters like 'vanilla' and 'chocolate'. Overall, experts and trained panellists were able to distinguish regions based on a few characteristic sensory traits.

Estimating minimal clinically important differences for two scales in patients with chronic traumatic brain injury

BACKGROUND

This study aimed to establish the minimal clinically important difference (MCID) for the Fugl-Meyer Motor Scale (FMMS) and the Disability Rating Scale (DRS) to evaluate interventions in patients with motor deficits in the chronic phase after traumatic brain injury (TBI).

METHODS

MCIDs were established with a structured expert consultation process, the RAND/UCLA modified Delphi method. This process consisted of a literature review and input from a 10-person, multidisciplinary expert panel. The experts were asked to rate meaningfulness of improvements in hypothetical patients and numeric changes via two rounds of ratings and an in-person meeting.

RESULTS

The estimated MCIDs were six and five points on the FMMS Upper and Lower Extremity Scale, respectively, and one point on the DRS. The experts argued against establishing an MCID for the combined FMMS because the same change was more likely to be meaningful if concentrated in one extremity and because a meaningful improvement in one extremity implies meaningfulness irrespective of the changes in the other.

CONCLUSIONS

This study is the first to establish MCIDs for the FMMS and the DRS in the chronic phase after TBI. The results may be helpful for the design and interpretation of clinical trials of interventions.

Development of a drinking water quality index based on a participatory procedure using mixed multicriteria methods

The objective of this study is to propose and apply a novel approach to develop a drinking water quality index that enables water managers to routinely identify the vulnerabilities in a distribution system (DS) while taking into account the priorities of water managers and operators. Here, we propose an innovative approach based on the participation of eight researchers and water managers. The input from these participants was included through a combination of the Delphi and MACBETH (Measuring Attractiveness by a Categorical-Based Evaluation Technique) methods, which were applied to an index developed using fuzzy logic. The index was then validated by conducting sensitivity and comparative analyses on a database from field sampling campaigns conducted in various small municipal DSs in two Canadian provinces: Quebec and Newfoundland and Labrador. Finally, the drinking water quality index was applied to a database of the DS in Quebec City, Canada. Sensitivity and comparative analyses revealed that the developed index could allow for the characterization of water sample quality and could prove useful for prioritizing interventions in the DS. The index was also useful for representing the spatial variability of water quality, and provided nuanced information about water quality, especially when the water quality parameter values complied with guidelines and regulations. These results provide a transferable and integrated approach for developing customized drinking water quality indices. This study will help pave the way for water quality managers to better prevent episodes of possible water quality deterioration.

Expert-based development of a generic HACCP-based risk management system to prevent critical negative energy balance in dairy herds

The objective of this study was to develop a generic risk management system based on the Hazard Analysis and Critical Control Point (HACCP) principles for the prevention of critical negative energy balance (NEB) in dairy herds using an expert panel approach. In addition, we discuss the advantages and limitations of the system in terms of implementation in the individual dairy herd. For the expert panel, we invited 30 researchers and advisors with expertise in the field of dairy cow feeding and/or health management from eight European regions. They were invited to a Delphi-based set-up that included three inter-correlated questionnaires in which they were asked to suggest risk factors for critical NEB and to score these based on 'effect' and 'probability'. Finally, the experts were asked to suggest critical control points (CCPs) specified by alarm values, monitoring frequency and corrective actions related to the most relevant risk factors in an operational farm setting. A total of 12 experts (40 %) completed all three questionnaires. Of these 12 experts, seven were researchers and five were advisors and in total they represented seven out of the eight European regions addressed in the questionnaire study. When asking for suggestions on risk factors and CCPs, these were formulated as 'open questions', and the experts' suggestions were numerous and overlapping. The suggestions were merged via a process of linguistic editing in order to eliminate doublets. The editing process revealed that the experts provided a total of 34 CCPs for the 11 risk factors they scored as most important. The consensus among experts was relatively high when scoring the most important risk factors, while there were more diverse suggestions of CCPs with specification of alarm values and corrective actions. We therefore concluded that the expert panel approach only partly succeeded in developing a generic HACCP for critical NEB in dairy cows. We recommend that the output of this paper is used to inform key areas for implementation on the individual dairy farm by local farm teams including farmers and their advisors, who together can conduct herd-specific risk factor profiling, organise the ongoing monitoring of herd-specific CCPs, as well as implement corrective actions when CCP alarm values are exceeded.

[Precision medicine in hematological malignancies]

Precision medicine is a type of medical care designed to optimize the therapeutic efficiency or benefit for particular groups of patients with the use of genetic profiling. The application of precision medicine in cancer treatment is prospected because cancer is reported to be the leading cause of death in Japan. Consequently, Japanese cancer genome medicine will be launched within this fiscal year. In this study, we focus on precision medicine specifically in the field of hematological malignancies with an overview of its clinical utility. We further discuss how precision medicine should be developed in this field, based on our experience of a feasibility study for clinical sequencing in hematological malignancies.

Use of an expert panel to identify domains and indicators of delirium severity

PURPOSE

Our purpose was to create a content domain framework for delirium severity to inform item development for a new instrument to measure delirium severity.

METHODS

We used an established, multi-stage instrument development process during which expert panelists discussed best approaches to measure delirium severity and identified related content domains. We conducted this work as part of the Better ASsessment of ILlness (BASIL) study, a prospective, observational study aimed at developing and testing measures of delirium severity. Our interdisciplinary expert panel consisted of twelve national delirium experts and four expert members of the core research group. Over a one-month period, experts participated in two rounds of review.

RESULTS

Experts recommended that the construct of delirium severity should reflect both the phenomena and the impact of delirium to create an accurate, patient-centered instrument useful to interdisciplinary clinicians and family caregivers. Final content domains were Cognitive, Level of consciousness, Inattention, Psychiatric-Behavioral, Emotional dysregulation, Psychomotor features, and Functional. Themes debated by experts included reconciling clinical geriatrics and psychiatric content, mapping symptoms to one specific domain, and accurate capture of unclear clinical presentations.

CONCLUSIONS

We believe this work represents the first application of instrument development science to delirium. The identified content domains are inclusive of various, wide-ranging domains of delirium severity and are reflective of a consistent framework that relates delirium severity to potential clinical outcomes. Our content domain framework provides a foundation for development of delirium severity instruments that can help improve care and quality of life for patients with delirium.

Informing the American Academy of family Physician's Health Equity strategy - an environmental scan using the Delphi technique

BACKGROUND

Many organizations have prioritized health equity and the social determinants of health (SDoH). These organizations need information to inform their planning, but, relatively few quantifiable measures exist. This study was conducted as an environmental scan to inform the American Academy of Family Physician's (AAFP's) health equity strategy. The objectives of the study were to identify and prioritize a comprehensive list of strategies in four focus areas: health equity leadership, policy, research, and diversity.

METHODS

A Delphi study was used to identify and prioritize the most important strategies for reducing health inequities among the four aforementioned focus areas. Health equity experts were purposefully sampled. Data were collected in three rounds for each focus area separately. A comprehensive list of strategy statements was identified for each focus area in round one. The strategy statements were prioritized in round two and reprioritized in a final third round. Quantitative and qualitative data were integrated for the final analysis.

RESULTS

Fifty strategies were identified across the four focus areas. Commitment to health equity, knowledge of health inequities, and knowledge of effective strategies to address the drivers of health inequities were ranked the highest for leadership. Universal access to health care and health in all policies were ranked highest for policy. Multi-level interventions, the effect of policy, governance, and politics, and translating and disseminating health equity interventions into practice were ranked the highest for research. Providing financial support to students from minority or low-socioeconomic backgrounds, commitment from undergraduate and medical school leadership for educational equity, providing opportunities for students from minority or low-socioeconomic backgrounds to prepare for standardized tests, and equitable primary and secondary school funding were ranked highest for diversity.

CONCLUSIONS

The AAFP and other medical specialty societies have an important opportunity to advance health equity. They should develop a health equity policy agenda, equip physicians and other stakeholders, use their connections with practice-based research networks to identify and translate practical solutions to address the SDoH, and advocate for a more diverse medical workforce.

TRIAL REGISTRATION

Not applicable.

Clinical use of [TIMP-2]•[IGFBP7] biomarker testing to assess risk of acute kidney injury in critical care: guidance from an expert panel

Background

The first FDA-approved test to assess risk for acute kidney injury (AKI), [TIMP-2]•[IGFBP7], is clinically available in many parts of the world, including the USA and Europe. We sought to understand how the test is currently being used clinically.

Methods

We invited a group of experts knowledgeable on the utility of this test for kidney injury to a panel discussion regarding the appropriate use of the test. Specifically, we wanted to identify which patients would be appropriate for testing, how the results are interpreted, and what actions would be taken based on the results of the test. We used a modified Delphi method to prioritize specific populations for testing and actions based on biomarker test results. No attempt was made to evaluate the evidence in support of various actions however.

Results

Our results indicate that clinical experts have developed similar practice patterns for use of the [TIMP-2]•[IGFBP7] test in Europe and North America. Patients undergoing major surgery (both cardiac and non-cardiac), those who were hemodynamically unstable, or those with sepsis appear to be priority patient populations for testing kidney stress. It was agreed that, in patients who tested positive, management of potentially nephrotoxic drugs and fluids would be a priority. Patients who tested negative may be candidates for “fast-track” protocols.

Conclusion

In the experience of our expert panel, biomarker testing has been a priority after major surgery, hemodynamic instability, or sepsis. Our panel members reported that a positive test prompts management of nephrotoxic drugs as well as fluids, while patients with negative results are considered to be excellent candidates for “fast-track” protocols.

Electronic supplementary material

The online version of this article (10.1186/s13054-019-2504-8) contains supplementary material, which is available to authorized users.

Expert panel process to optimise the design of a randomised controlled trial in chronic rhinosinusitis (the MACRO programme)

BACKGROUND

MACRO (Defining best Management for Adults with Chronic RhinOsinusitis) is an NIHR-funded programme of work designed to establish best practice for adults with chronic rhinosinusitis (CRS). The 7-year programme comprises three consecutive workstreams, designed to explore NHS care pathways through analysis of primary and secondary data sources, and to undertake a randomised controlled trial to evaluate a longer-term course of macrolide antibiotics and endoscopic sinus surgery for patients with CRS. A number of outstanding elements still required clarification at the funding stage. This paper reports an expert panel review process designed to agree and finalise the MACRO trial design, ensuring relevance to patients and clinicians whilst maximising trial recruitment and retention.

METHODS

An expert panel, consisting of the MACRO Programme Management Group, independent advisors, and patient contributors, was convened to review current evidence and the mixed-method data collected as part of the programme, and reach agreement on MACRO trial design. Specifically, agreement was sought for selection of macrolide antibiotic, use of orally administered steroids, inclusion of CRS phenotypes (with/without nasal polyps), and overall trial design.

RESULTS

A 12-week course of clarithromycin was agreed as the main trial comparator due to its increasing use as a first- and second-line treatment for patients with CRS, and the perceived need to establish its role in CRS management. Orally administered steroids will be used as a rescue medication during the trial, rather than routinely either pre or post trial randomisation, to limit any potential effects on surgical outcomes and better reflect current UK prescribing habits. Both CRS phenotypes will be included in a single trial to ensure that the MACRO trial is both pragmatic and generalisable to primary care. A modified, three-arm trial design was agreed after intense discussions and further exploratory work. Inclusion criteria were amended to ensure that the patients recruited would be considered eligible for the treatment offered in the trial due to having already received appropriate medical therapy as deemed suitable by their ENT surgeon. A proposed 6-week run-in period prior to randomisation was removed due to the new criteria prior to randomisation.

CONCLUSION

The expert panel review process resulted in agreement on key elements and an optimal design for the MACRO trial, considered most likely to be successful in terms of both recruitment potential and ability to establish best management of patients with CRS.

Expert panel diagnosis demonstrated high reproducibility as reference standard in infectious diseases

OBJECTIVE

If a gold standard is lacking in a diagnostic test accuracy study, expert diagnosis is frequently used as reference standard. However, interobserver and intraobserver agreements are imperfect. The aim of this study was to quantify the reproducibility of a panel diagnosis for pediatric infectious diseases.

STUDY DESIGN AND SETTING

Pediatricians from six countries adjudicated a diagnosis (i.e., bacterial infection, viral infection, or indeterminate) for febrile children. Diagnosis was reached when the majority of panel members came to the same diagnosis, leaving others inconclusive. We evaluated intraobserver and intrapanel agreement with 6 weeks and 3 years' time intervals. We calculated the proportion of inconclusive diagnosis for a three-, five-, and seven-expert panel.

RESULTS

For both time intervals (i.e., 6 weeks and 3 years), intrapanel agreement was higher (kappa 0.88, 95%CI: 0.81-0.94 and 0.80, 95%CI: NA) compared to intraobserver agreement (kappa 0.77, 95%CI: 0.71-0.83 and 0.65, 95%CI: 0.52-0.78). After expanding the three-expert panel to five or seven experts, the proportion of inconclusive diagnoses (11%) remained the same.

CONCLUSION

A panel consisting of three experts provides more reproducible diagnoses than an individual expert in children with lower respiratory tract infection or fever without source. Increasing the size of a panel beyond three experts has no major advantage for diagnosis reproducibility.

Definition of treatment-resistant depression - Asia Pacific perspectives

BACKGROUND

The lack of uniformity in the definition of treatment resistant depression (TRD) within the Asia-Pacific (APAC) region may have implications for patient management. We aimed to characterize the most commonly used TRD definition in selected APAC countries.

METHODS

A systematic literature review of TRD definitions in APAC countries was conducted in Medline and Embase (2010-2016) and conference proceedings (2014 and 2016). TRD guidelines (APAC, Europe regional, US, or international) were also searched. An expert-panel explored APAC nuances in TRD definitions to achieve consensus for a regional-level definition.

RESULTS

Ten guidelines and 89 studies qualified for study inclusion. Among the studies, variations were observed in definitions regarding: number of antidepressants failed (range: ≥1 to ≥3), classes of antidepressants (same or different; 59% did not specify class), duration of previous treatments (range: 4-12 weeks), dosage adequacy, and consideration of adherence (yes/no; 88% of studies did not consider adherence). No TRD-specific guidelines were identified. The emerging consensus from the literature review and panel discussion was that TRD is most commonly defined as failure to ≥2 antidepressant therapies given at adequate doses, for 6-8 weeks during a major depressive episode.

LIMITATIONS

Few studies provided definitions of TRD used in daily clinical practice, and a limited number of countries were represented in the included studies and expert panel.

CONCLUSION

Attaining consensus on TRD definition may promote accurate, and possibly early detection of patients with TRD to enable appropriate intervention that may impact patient outcomes and quality of life.

Use of magnetic resonance imaging in rectal cancer patients: Society of Abdominal Radiology (SAR) rectal cancer disease-focused panel (DFP) recommendations 2017

PURPOSE

To propose guidelines based on an expert-panel-derived unified approach to the technical performance, interpretation, and reporting of MRI for baseline and post-treatment staging of rectal carcinoma.

METHODS

A consensus-based questionnaire adopted with permission and modified from the European Society of Gastrointestinal and Abdominal Radiologists was sent to a 17-member expert panel from the Rectal Cancer Disease-Focused Panel of the Society of Abdominal Radiology containing 268 question parts. Consensus on an answer was defined as ≥ 70% agreement. Answers not reaching consensus (< 70%) were noted.

RESULTS

Consensus was reached for 87% of items from which recommendations regarding patient preparation, technical performance, pulse sequence acquisition, and criteria for MRI assessment at initial staging and restaging exams and for MRI reporting were constructed.

CONCLUSION

These expert consensus recommendations can be used as guidelines for primary and post-treatment staging of rectal cancer using MRI.

Lack of preregistered analysis plans allows unacceptable data mining for and selective reporting of consensus in Delphi studies

OBJECTIVES

To empirically demonstrate how undisclosed analytic flexibility provides substantial latitude for data mining and selective reporting of consensus in Delphi processes.

STUDY DESIGN AND SETTING

Pooling data across eight online modified-Delphi panels, we first calculated the percentage of items reaching consensus according to descriptive analysis procedures commonly used in health research but selected post hoc in this article. We then examined the variability of items reaching consensus across panels.

RESULTS

Pooling all panel data, the percentage of items reaching consensus ranged from 0% to 84%, depending on the analysis procedure. Comparing data across panels, variability in the percentage of items reaching consensus for each analysis procedure ranged from 0 (i.e., all panels had the same percentage of items reaching consensus for a given analysis procedure) to 83 (i.e., panels had a range of 11% to 94% of items reaching consensus for a given analysis procedure). Of 200 total panel-by-analysis-procedure configurations, four configurations (2%) had all items and 64 (32%) had no items reaching consensus.

CONCLUSION

Undisclosed analytic flexibility makes it unacceptably easy to data mine for and selectively report consensus in Delphi processes. As a solution, we recommend prospective, complete registration of preanalysis plans for consensus-oriented Delphi processes in health research.

A quick and selected overview of the expert panel on effective ways of investing in health

The European Commission created the Expert Panel on Effective Ways of Investing in Health (EXPH) in 2012. The EXPH started its activities in July 2013 and ended its first term in May 2016. A personal review of the Expert Panel contributions in its first term is provided.

Clinical photographic observation of plantar corns and callus associated with a nominal scale classification and inter- observer reliability study in a student population

BACKGROUND

The management of plantar corns and callus has a low cost-benefit with reduced prioritisation in healthcare. The distinction between types of keratin lesions that forms corns and callus has attracted limited interest. Observation is imperative to improving diagnostic predictions and a number of studies point to some confusion as to how best to achieve this. The use of photographic observation has been proposed to improve our understanding of intractable keratin lesions.

METHODS

Students from a podiatry school reviewed photographs where plantar keratin lesions were divided into four nominal groups; light callus (Grade 1), heavy defined callus (Grade 2), concentric keratin plugs (Grade 3) and callus with deeper density changes under the forefoot (Grade 4). A group of 'experts' assigned from qualified podiatrists validated the observer rated responses by the students.

RESULTS

Cohen's weighted statistic (k) was used to measure inter-observer reliability. First year students (unskilled) performed less well when viewing photographs (k = 0.33) compared to third year students (semi-skilled, k = 0.62). The experts performed better than students (k = 0.88) providing consistency with wound care models in other studies.

CONCLUSIONS

Improved clinical annotation of clinical features, supported by classification of keratin- based lesions, combined with patient outcome tools, could improve the scientific rationale to prioritise patient care. Problems associated with photographic assessment involves trying to differentiate similar lesions without the benefit of direct palpation. Direct observation of callus with and without debridement requires further investigation alongside the model proposed in this paper.

PrediCTC, liquid biopsy in precision oncology: a technology transfer experience in the Spanish health system

PURPOSE

Management of metastatic disease in oncology includes monitoring of therapy response principally by imaging techniques like CT scan. In addition to some limitations, the irruption of liquid biopsy and its application in personalized medicine has encouraged the development of more efficient technologies for prognosis and follow-up of patients in advanced disease.

METHODS

PrediCTC constitutes a panel of genes for the assessment of circulating tumor cells (CTC) in metastatic colorectal cancer patients, with demonstrated improved efficiency compared to CT scan for the evaluation of early therapy response in a multicenter prospective study. In this work, we designed and developed a technology transfer strategy to define the market opportunity for an eventual implementation of PrediCTC in the clinical practice.

RESULTS

This included the definition of the regulatory framework, the analysis of the regulatory roadmap needed for CE mark, a benchmarking study, the design of a product development strategy, a revision of intellectual property, a cost-effectiveness study and an expert panel consultation.

CONCLUSION

The definition and analysis of an appropriate technology transfer strategy and the correct balance among regulatory, financial and technical determinants are critical for the transformation of a promising technology into a viable technology, and for the decision of implementing liquid biopsy in the monitoring of therapy response in advanced disease.

Follow-up intervals in patients with Cushing's disease: recommendations from a panel of experienced pituitary clinicians

PURPOSE

Follow-up guidelines are needed to assess quality of care and to ensure best long-term outcomes for patients with Cushing's disease (CD). The purpose of this study was to assess agreement by experts on recommended follow-up intervals for CD patients at different phases in their treatment course.

METHODS

The RAND/UCLA modified Delphi process was used to assess expert consensus. Eleven clinicians who regularly manage CD patients rated 79 hypothetical patient scenarios before and after ("second round") an in-person panel discussion to clarify definitions. Scenarios described CD patients at various time points after treatment. For each scenario, panelists recommended follow-up intervals in weeks. Panel consensus was assigned as follows: "agreement" if no more than two responses were outside a 2 week window around the median response; "disagreement" if more than two responses were outside a 2 week window around the median response. Recommendations were developed based on second round results.

RESULTS

Panel agreement was 65.9% before and 88.6% after the in-person discussion. The panel recommended follow-up within 8 weeks for patients in remission on glucocorticoid replacement and within 1 year of surgery; within 4 weeks for patients with uncontrolled persistent or recurrent disease; within 8-24 weeks in post-radiotherapy patients controlled on medical therapy; and within 24 weeks in asymptomatic patients with stable plasma ACTH concentrations after bilateral adrenalectomy.

CONCLUSIONS

With a high level of consensus using the Delphi process, panelists recommended regular follow-up in most patient scenarios for this chronic condition. These recommendations may be useful for assessment of CD care both in research and clinical practice.

Quality of care measures for the management of unhealthy alcohol use

There is a paucity of quality measures to assess the care for the range of unhealthy alcohol use, ranging from risky drinking to alcohol use disorders. Using a two-phase expert panel review process, we sought to develop an expanded set of quality of care measures for unhealthy alcohol use, focusing on outpatient care delivered in both primary care and specialty care settings. This process generated 25 candidate measures. Eight measures address screening and assessment, 11 address aspects of treatment, and six address follow-up. These quality measures represent high priority targets for future development, including creating detailed technical specifications and pilot testing them to evaluate their utility in terms of feasibility, reliability, and validity.

Creating authentic video scenarios for use in prehospital research

Video scenarios have been used to explore clinical reasoning during interviews in Think Aloud studies. This study used nominal group technique with experts to create video scenarios to explore the ways paramedics think and reason when caring for children who are sick or injured. At present there is little research regarding paramedics' clinical reasoning with respect to performing non-urgent procedures on children. A core expert panel identified the central structure of a prehospital clinical interaction and the range of contextual factors that may influence a paramedic's clinical reasoning [the way in which information is gathered, interpreted and analysed by clinicians]. The structure and contextual factors were then incorporated into two filmed scenarios. A second panel of clinical practice experts, then critiqued the body language, spoken word and age appropriate behaviours of those acting in the video scenarios and compared them against their own experience of clinical practice to confirm authenticity. This paper reports and reflects on the use of nominal group technique to create authentic video scenarios for use in prehospital research.

An expert panel process to evaluate habitat restoration actions in the Columbia River estuary

We describe a process for evaluating proposed ecosystem restoration projects intended to improve survival of juvenile salmon in the Columbia River estuary (CRE). Changes in the Columbia River basin (northwestern USA), including hydropower development, have contributed to the listing of 13 salmon stocks as endangered or threatened under the U.S. Endangered Species Act. Habitat restoration in the CRE, from Bonneville Dam to the ocean, is part of a basin-wide, legally mandated effort to mitigate federal hydropower impacts on salmon survival. An Expert Regional Technical Group (ERTG) was established in 2009 to improve and implement a process for assessing and assigning "survival benefit units" (SBUs) to restoration actions. The SBU concept assumes site-specific restoration projects will increase juvenile salmon survival during migration through the 234 km CRE. Assigned SBUs are used to inform selection of restoration projects and gauge mitigation progress. The ERTG standardized the SBU assessment process to improve its scientific integrity, repeatability, and transparency. In lieu of experimental data to quantify the survival benefits of individual restoration actions, the ERTG adopted a conceptual model composed of three assessment criteria-certainty of success, fish opportunity improvements, and habitat capacity improvements-to evaluate restoration projects. Based on these criteria, an algorithm assigned SBUs by integrating potential fish density as an indicator of salmon performance. Between 2009 and 2014, the ERTG assessed SBUs for 55 proposed projects involving a total of 181 restoration actions located across 8 of 9 reaches of the CRE, largely relying on information provided in a project template based on the conceptual model, presentations, discussions with project sponsors, and site visits. Most projects restored tidal inundation to emergent wetlands, improved riparian function, and removed invasive vegetation. The scientific relationship of geomorphic and salmonid responses to restoration actions remains the foremost concern. Although not designed to establish a broad strategy for estuary restoration, the scoring process has adaptively influenced the types, designs, and locations of restoration proposals. The ERTG process may be a useful model for others who have unique ecosystem restoration goals and share some of our common challenges.

Patient safety priorities in mental healthcare in Switzerland: a modified Delphi study

OBJECTIVE

Identifying patient safety priorities in mental healthcare is an emerging issue. A variety of aspects of patient safety in medical care apply for patient safety in mental care as well. However, specific aspects may be different as a consequence of special characteristics of patients, setting and treatment. The aim of the present study was to combine knowledge from the field and research and bundle existing initiatives and projects to define patient safety priorities in mental healthcare in Switzerland. The present study draws on national expert panels, namely, round-table discussion and modified Delphi consensus method.

DESIGN

As preparation for the modified Delphi questionnaire, two round-table discussions and one semistructured questionnaire were conducted. Preparative work was conducted between May 2015 and October 2015. The modified Delphi was conducted to gauge experts' opinion on priorities in patient safety in mental healthcare in Switzerland. In two independent rating rounds, experts made private ratings. The modified Delphi was conducted in winter 2015.

RESULTS

Nine topics were defined along the treatment pathway: diagnostic errors, non-drug treatment errors, medication errors, errors related to coercive measures, errors related to aggression management against self and others, errors in treatment of suicidal patients, communication errors, errors at interfaces of care and structural errors.

CONCLUSIONS

Patient safety is considered as an important topic of quality in mental healthcare among experts, but it has been seriously neglected up until now. Activities in research and in practice are needed. Structural errors and diagnostics were given highest priority. From the topics identified, some are overlapping with important aspects of patient safety in medical care; however, some core aspects are unique.

Social choice functions: A tool for ranking variables involved in action plans against road noise

Traffic noise is gaining importance in planning and operation of roads in developing countries, and particularly in Europe and Latin America. Many variables with different degrees of importance influence the perception of noise from roads. Thus, the problem of prioritizing road stretches for action against such noise is an important issue in environmental noise management. For example, it can be addressed using multicriteria methods. However, these methodologies require criteria or suitable variables to be ranked according to their relative importance. In the present study, for this ranking, a list of nine variables involved in the decision-making process (called "road stretch priority variables") was presented in the form of questionnaires to high-level experts from Andalusia, southern Spain. These experts ranked the variables by relevance. Using the same data, seven social choice functions (Plurality, Raynaud, Kemeny-Young, Copeland, Simpson, Schulze, and Borda) were used in order to rank the variables. The results indicate that the most important variables were those that take into account the parameters of greatest exposure for the citizens, followed by variables related to the intensity of the problem analyzed. The results show that a combination of the use of social choice functions on aggregated information from expert panels can provide a consensus for ranking priority variables related to road stretches.

Performing Lieder: Expert Perspectives and Comparison of Vibrato and Singer's Formant With Opera Singers

This article reports three studies about performance of lieder, and in particular in comparison with opera performance. In study 1, 21 participants with experience in music performance and teaching completed a survey concerning various characteristics of lieder performance. The results showed that there was consensus between the literature and the assessment of an expert panel-that a "natural" and "unoperatic" vibrato was favored, and that diction, text, and variation of tone are all important aspects of lieder performance. Two acoustic analyses were conducted to investigate genre-specific differences of the singer's formant and vibrato parameters. The first analysis (study 2) used 18 single quasi-unaccompanied notes from commercial recordings of two lieder, and, for comparison, 20 single unaccompanied notes from an opera. Vibrato rate was statistically identical between the two genres at ~6.4 Hz; however, lieder featured a longer delay in vibrato onset. Vibrato extent was smaller for lieder (~112 cents) compared with opera (~138 cents). The singer's formant, which is generally associated with opera, was at times observed in the lieder recordings; however, this was at an overall significantly weaker intensity than in the opera recordings. The results were replicated in study 3, where recordings using only singers who performed in both lied and opera were analyzed. This direct comparison used 45 lieder notes and 55 opera notes and also investigated three different methods of analyzing the singer's formant. A number of consistencies and inconsistencies were identified between acoustic parameters reported in studies 2 and 3, and the beliefs of singing teachers and scholars in the literature and study 1.

Developing an expert panel process to refine health outcome definitions in observational data

OBJECTIVES

Drug safety surveillance using observational data requires valid adverse event, or health outcome of interest (HOI) measurement. The objectives of this study were to develop a method to review HOI definitions in claims databases using (1) web-based digital tools to present de-identified patient data, (2) a systematic expert panel review process, and (3) a data collection process enabling analysis of concepts-of-interest that influence panelists' determination of HOI.

METHODS

De-identified patient data were presented via an interactive web-based dashboard to enable case review and determine if specific HOIs were present or absent. Criteria for determining HOIs and their severity were provided to each panelist. Using a modified Delphi method, six panelist pairs independently reviewed approximately 200 cases across each of three HOIs (acute liver injury, acute kidney injury, and acute myocardial infarction) such that panelist pairs independently reviewed the same cases. Panelists completed an assessment within the dashboard for each case that included their assessment of the presence or absence of the HOI, HOI severity (if present), and data contributing to their decision. Discrepancies within panelist pairs were resolved during a consensus process.

RESULTS

Dashboard development was iterative, focusing on data presentation and recording panelists' assessments. Panelists reported quickly learning how to use the dashboard. The assessment module was used consistently. The dashboard was reliable, enabling an efficient review process for panelists. Modifications were made to the dashboard and review process when necessary to facilitate case review. Our methods should be applied to other health outcomes of interest to further refine the dashboard and case review process.

CONCLUSION

The expert review process was effective and was supported by the web-based dashboard. Our methods for case review and classification can be applied to future methods for case identification in observational data sources.

Expert panel assessment of acute liver injury identification in observational data

BACKGROUND

Observational data are useful for studying drug safety; however, to be effective, accurate outcome measurement is paramount.

OBJECTIVES

This study compared alternative outcome definitions for acute liver injury (ALI) and explored opportunities for improving ALI identification in observational data.

METHODS

The Truven MarketScan® Lab Database (MSLR) was used to identify patients meeting at least 1 of 4 ALI definitions, including definitions based on diagnosis codes, laboratory measures, or combinations of diagnoses, procedures, and/or laboratory measures. Expert panelists reviewed patient data using a Web dashboard. Panelists determined whether they believed the patient had ALI and identified factors influencing their decision. Logistic regression models explored which factors were influential in case determination.

RESULTS

Overall, only 37 of 208 reviewed patients (17.8%) were classified as cases. The diagnosis-based definition yielded no positive cases and the laboratory-based definition yielded the most positive cases (31 of 60). The most influential factors in case classification were occurrence of procedures after the index date (OR = 13.2, 95% CI = 5.3-32.9), no occurrence of drug treatments before the index date (OR = 4.6; 95% CI = 1.6-13.2), occurrence of drug treatments before the index date (OR = 0.3; 95% CI = 0.1-0.6), and no drug treatments after the index date (OR = 0.2; 95% CI = 0.0-0.5).

CONCLUSIONS

Comparing ALI definitions illustrated tradeoffs between the number of plausible cases identified and the likelihood of cases being classified as positive. Future research should refine ALI case definitions, considering the import of laboratory results, procedures, and drugs in defining a case.