How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings?

Integrating Patient and Expert Perspectives to Conceptualize High-Quality Palliative Cancer Care for Symptoms in the US Veterans Health Administration: A Qualitative Study

Quality measurement is typically the domain of clinical experts and health system leaders; patient/caregiver perspectives are rarely solicited. We aimed to describe and integrate clinician and patient/caregiver conceptualizations of high-quality palliative symptom care for patients receiving care for advanced cancer within the US Veterans Health Administration in the context of existing quality measures. We conducted a secondary qualitative analysis of transcripts from prioritization discussions of process quality measures relevant to cancer palliative care. These discussions occurred during 2 modified RAND-UCLA appropriateness panels: a panel of 10 palliative care clinical expert stakeholders (7 physicians, 2 nurses, 1 social worker) and a panel of 9 patients/caregivers with cancer experience. Discussions were recorded, transcribed, and independently double-coded using an a priori logical framework. Content analysis was used to identify subthemes within codes and axial coding was used to identify crosscutting themes. Patients/caregivers and clinical experts contributed important perspectives to 3 crosscutting themes. First, proactive elicitation of symptoms is critical. Patients/caregivers especially emphasized importance of comprehensive and proactive screening and assessment, especially for pain and mental health. Second, screening and assessment alone is not enough; information elicited from patients must inform care. Measuring screening/assessment and management care processes separately has important limitations. Lastly, high-quality symptom management can be broadly defined if it is patient-centered; high-quality care takes an individualized approach and might include non-medical or non-pharmacological symptom management. Integrating the perspectives of clinical experts and patients/caregivers is critical for health systems to consider as they design and implement quality measures for palliative cancer care.

Reporting guidelines for Delphi techniques in health sciences: A methodological review

BACKGROUND

Delphi techniques are conducted across different subfields in the health sciences. The reporting practices of studies using Delphi techniques vary, and current reporting guidelines for Delphi techniques focus on individual subfields of the health sciences or on different aspects of research and are therefore of limited applicability. The aim of this article was to identify similarities, differences, and possible shortcomings of existing Delphi reporting guidelines and to draft an initial proposal for a comprehensively applicable reporting guideline.

METHODS

A systematic literature search for reporting guidelines on Delphi studies was performed in existing data resources based on databases in the health sciences (Scopus, MEDLINE, CINAHL, Epistemonikos) including publications from 2016 to 2021. In June 2021, we conducted an additional search in PubMed and included further studies by contacting experts of the scientific Delphi expert network (DeWiss). Title and abstract screening of articles was performed, followed by a full-text screening of the articles included. We qualitatively and quantitatively evaluated, compared and contrasted the reporting guidelines identified using content analysis and discussed the results among the members of the Delphi expert network.

RESULTS

We retrieved ten health science articles with reporting guidelines for Delphi studies. In analyzing them, we identified nine main categories (Justification, Expert panel, Questionnaire, Survey design, Process regulation, Analyses, Results, Discussion, Methods reflection & Ethics). The current reporting guidelines vary significantly, with only the aspect of consensus appearing in all of them. Frequency distributions show that most of the subcategories are only addressed in individual articles (e.g., meeting of participants, proceeding with the survey method, transfer of the results, validation, prevention of bias) and that epistemological foundations of the Delphi technique are rarely mentioned or reflected on. We drafted an initial proposal for Delphi reporting guidelines for the health science sector.

DISCUSSION

A well-justified position concerning epistemological foundations of Delphi studies is necessary to make the quality of the process assessable and, along with the reporting of the process, to classify and compare study results. This will increase the acceptance of both the method in the health science sector and the results in medical practice. A Delphi reporting guideline must, above all, take into account the diversity of variants, subfield-related objectives and application areas, and their modifications of the Delphi technique in order to be comprehensively applicable in the health sciences.

CONCLUSION

The results of our methodological review do not provide a final reporting guideline. The newly developed proposal is intended to encourage discussion and agreement in further analyses.

Intervention elements and behavior change techniques to improve prescribing for older adults with multimorbidity in Singapore: a modified Delphi study

PURPOSE

Potentially inappropriate prescribing among older adults is a rising concern, attributed mainly by polypharmacy and multimorbidity. We aimed to identify key components and strategies for construction of a context-relevant intervention to facilitate appropriate prescribing in outpatient clinics in Singapore.

METHODS

The modified Delphi study was conducted in two rounds with 20 geriatricians from seven public hospitals in Singapore. Round one survey presented 69 statements formulated from a scoping review, while round two presented 23 statements with some modifications based on round one comments. The statements were rated against a 7-point Likert scale on their importance and impact on prescribing for older adults with multimorbidty.

RESULTS

Consensus were achieved for 90% of the statements. Seven intervention elements were identified as being important: medication review, training, medication therapy management, shared decision making, patient interview, medication reconciliation, comprehensive geriatric assessment. In addition, some commonly identified behavior change techniques included goal setting (behavior), goal setting (outcome) and problem solving.

CONCLUSIONS

This study identified important intervention elements and their potential strategies that could be adopted in an intervention to optimize appropriate prescribing for older adults with multimorbidity.

Coming to consensus: the Delphi technique

Delphi techniques are used in health care and nursing to systematically bring together explicit and implicit knowledge from experts with a research or practical background, often with the goal of reaching a group consensus. Consensus standards and findings are important for promoting the exchange of information and ideas on an interdisciplinary and transdisciplinary basis, and for guaranteeing comparable procedures in diagnostic and therapeutic approaches. Yet, the development of consensus standards using Delphi techniques is challenging because it is dependent on the willingness of experts to participate and the statistical definition of consensus.

Quality of care for people with multimorbidity: a focus group study with patients and their relatives

BACKGROUND

Prevalence of people with multimorbidity rises. Multimorbidity constitutes a challenge to the healthcare system, and treatment of patients with multimorbidity is prone to high-quality variations. Currently, no set of quality indicators (QIs) exists to assess quality of care, let alone incorporating the patient perspective. We therefore aim to identify aspects of quality of care relevant to the patients' perspective and match them to a literature-based set of QIs.

METHODS

We conducted eight focus groups with patients with multimorbidity and three focus groups with patients' relatives using a semistructured guide. Data were analysed using Kuckartz's qualitative content analysis. We derived deductive categories from the literature, added inductive categories (new quality aspects) and translated them into QI.

RESULTS

We created four new QIs based on the quality aspects relevant to patients/relatives. Two QIs (patient education/self-management, regular updates of medication plans) were consented by an expert panel, while two others were not (periodical check-ups, general practitioner-coordinated care). Half of the literature-based QIs, for example, assessment of biopsychosocial support needs, were supported by participants' accounts, while more technical domains regarding assessment and treatment regimens were not addressed in the focus groups.

CONCLUSION

We show that focus groups with patients and relatives adding relevant aspects in QI development should be incorporated by default in QI development processes and constitute a reasonable addition to traditional QI development. Our QI set constitutes a framework for assessing the quality of care in the German healthcare system. It will facilitate implementation of treatment standards and increase the use of existing guidelines, hereby helping to reduce overuse, underuse and misuse of healthcare resources in the treatment of patients with multimorbidity.

TRIAL REGISTRATION NUMBER

German clinical trials registry (DRKS00015718), Pre-Results.

Enhancing organizational readiness for implementation: constructing a typology of readiness-development strategies using a modified Delphi process

BACKGROUND

Knowledge about the development of organizational readiness for implementation (ORI) is limited. ORI, referred to as the willingness and capacity of all relevant stakeholders to change practice, is critical for increasing the adoption rate of evidence-based practices and improving implementation outcomes. However, no methodology currently guides ORI's enhancement or addresses differences in readiness needs across an organization. This study used the transtheoretical model (TTM) as a framework for classifying a well-established compilation of implementation strategies into three readiness stages: pre-contemplation, contemplation, and preparation.

METHODS

A modified Delphi method was used to establish consensus among a panel of purposefully selected research and field implementation experts. The Delphi process involved three rounds of online questionnaires. The third round also included a live video discussion to clarify definitions in an effort to increase consensus among experts.

RESULTS

Of the 73 strategies reviewed, the experts identified 75% (n = 55) as relevant for pre-implementation and reached a high-level agreement on the assignment of 7% (n = 5) of the strategies to the pre-contemplation stage (ORI-1), 25% (n = 18) to the contemplation stage (ORI-2), and 52% (n = 38) to the preparation stage (ORI-3). Several strategies were identified as relevant to more than one stage.

CONCLUSIONS

Participating experts were able to reach high-level agreement on the relevance of specific sets of implementation strategies to each of the three ORI stages. The lowest number of strategies was assigned to ORI-1 and the highest number to ORI-3. Given the overlap of strategies across ORI stages, there is a need to better understand the specific utilization of such strategies at different stages. Future studies are needed to empirically evaluate the relevance and applicability of this expert-informed typology based on implementers' experiences in the field.

Expert Stakeholder Prioritization of Process Quality Measures to Achieve Patient- and Family-Centered Palliative and End-of-Life Cancer Care

Importance: Quality measures of palliative and end-of-life care relevant to patients with advanced cancer have been developed, but few are in routine use. It is unclear which of these measures are most important for providing patient- and family-centered care and have high potential for improving quality of care. Objective: To prioritize process quality measures for assessing delivery of patient- and family-centered palliative and end-of-life cancer care in US Veterans Affairs (VA) health care facilities. Design, Setting, Participants: A panel of 10 palliative and cancer care expert stakeholders (7 physicians, 2 nurses, 1 social worker) rated process quality measure concepts before and after a 1-day meeting. Measures: Panelists rated 64 measure concepts on a nine-point scale on: (1) importance to providing patient- and family-centered care, and (2) potential for quality improvement (QI). Panelists also nominated five highest priority measure concepts ("top 5") on each attribute. Results: Panelists rated most measure concepts (54 premeeting, 56 post-meeting) as highly important to patient- and family-centered care (median rating ≥7). Considerably fewer (17 premeeting, 22 post-meeting) were rated as having high potential for QI. Measure concepts having postpanel median ratings ≥7 and nominated by one or more panelists as "top 5" on either attribute comprised a shortlist of 20 measure concepts. Conclusions: A panel of expert stakeholders helped prioritize 64 measure concepts into a shortlist of 20. Half of the shortlisted measures were related to communication about patient preferences and decision making, and half were related to symptom assessment and treatment.

Patient-Reported Outcome Measures in Mental Health Clinical Research: A Descriptive Review in Comparison with Clinician-Rated Outcome Measures

PURPOSE

To review how patient-reported outcomes measures in mental health clinical research complement traditional clinician-rated outcomes measures.

DATA SOURCES

Medline, Embase, PsycInfo and Scopus.

STUDY SELECTION

Latest update of the literature search was conducted in August 2019, using a specified set of search terms to identify controlled and uncontrolled studies (published since 1996) of pharmacological or non-pharmacological interventions in adults (≥18 years) in hospital-based mental health care.

DATA EXTRACTION

Two authors extracted data independently using a pre-designed extraction form.

RESULTS OF DATA SYNTHESIS

Among the 2962 publications identified, 257 were assessed by full text reading. A total of 24 studies reported in 26 publications were included in this descriptive review. We identified subjective and objective outcome measures, classified these according to the pharmacopsychometric triangle and compared them qualitatively in terms of incremental information added to the clinical study question. The data reviewed here from primarily depression and schizophrenia intervention studies show that results from patient-reported outcome measures and clinician-rated outcome measures generally point in the same direction. There was a relative lack of patient-reported outcome measures on functioning and medication side effects compared with patient-reported outcome measures on symptom burden and health-related quality of life.

CONCLUSION

Patient-reported outcomes and clinician-rate outcomes supplement each other and at most times support identical study conclusions. Future studies would benefit from a more systematic approach towards use of patient-reported outcomes and a clearer rationale of how to weigh and report the results in comparison with clinician-rated outcomes.

A Blueprint for Streamlining Patient Pathways Using a Hybrid Lean Management Approach

BACKGROUND

Streamlining patient pathways within health care systems is a complex and challenging process. While frontline clinicians often have an abundance of ideas, these rarely translate into real-world change due to nonadoption or early abandonment.

OBJECTIVES

The aim of this article is to provide frontline clinicians with a blueprint for developing a business case for a streamlined pathway while guiding the practical implementation of this blueprint.

METHODS

The key steps outlined in streamlining a patient pathway are as follows: step 1-identify problems with the patient pathway; step 2-identify the potential to streamline; step 3-forecast the benefits of the streamlined pathway; step 4-gain approvals; step 5-plan the practicalities; step 6-implement and monitor the streamlined pathway; and step 7-monitor the streamlined pathway. Within these steps, Lean management techniques are introduced (including value stream mapping, Pareto charts, Ishikawa diagrams, demand and capacity calculations, role lane mapping) and strengthened by other methods (retrospective audit, systematic review, patient questionnaires, and cost analysis).

RESULTS

This roadmap is contextualized using a case study, demonstrating how streamlining pathways can result in statistically significant reductions in referral to treatment time, the number of steps in the pathway, lead time (pathway duration), and handoff (transfer of patients between health care professionals). This can be achieved while increasing patient contact time, improving patient satisfaction, and reducing costs.

CONCLUSION

This blueprint demonstrates a comprehensive method for streamlining patient pathways, using Lean management techniques complemented by additional methods. This approach was developed by frontline clinicians and can be replicated by others, translating quality improvement ideas into sustainable change in practice. It enables the design of streamlined pathways that confer significant benefits to patients, health care service providers, and the health economy.

Delphi Technique in Health Sciences: A Map

Objectives: In health sciences, the Delphi technique is primarily used by researchers when the available knowledge is incomplete or subject to uncertainty and other methods that provide higher levels of evidence cannot be used. The aim is to collect expert-based judgments and often to use them to identify consensus. In this map, we provide an overview of the fields of application for Delphi techniques in health sciences in this map and discuss the processes used and the quality of the findings. We use systematic reviews of Delphi techniques for the map, summarize their findings and examine them from a methodological perspective.

Methods: Twelve systematic reviews of Delphi techniques from different sectors of the health sciences were identified and systematically analyzed.

Results: The 12 systematic reviews show, that Delphi studies are typically carried out in two to three rounds with a deliberately selected panel of experts. A large number of modifications to the Delphi technique have now been developed. Significant weaknesses exist in the quality of the reporting.

Conclusion: Based on the results, there is a need for clarification with regard to the methodological approaches of Delphi techniques, also with respect to any modification. Criteria for evaluating the quality of their execution and reporting also appear to be necessary. However, it should be noted that we cannot make any statements about the quality of execution of the Delphi studies but rather our results are exclusively based on the reported findings of the systematic reviews.

What are the main patient safety concerns of healthcare stakeholders: a mixed-method study of Web-based text

•

Due to the importance of safety in quality care, it’s national policy should be created using a bottom-up approach from various healthcare stakeholders.

•

To explore latent concerns of consumers, providers, government bodies, and researchers, text data analysis on patient safety collected from websites was useful for summarizing various aspects of concern.

•

A common concern among stakeholders was hospital infection control, ranging from nosocomial infections to those brought in by visiting patients around the Patient Safety Act legislation of Korea in 2015.

•

Researchers were focused on hospital sociocultural factors at both the organizational and clinician levels. Government policies and systemic approaches to patient safety were highlighted by different stakeholders.

•

Five topics including infection control showed statistically significant increasing trends over time, while another five showed decreasing trends.

Objectives

Various healthcare stakeholders define quality of care in different ways. Public policy could advocate all these concerns. This study was conducted to identify the main themes on patient safety of stakeholders expressed before and after the Patient Safety Act was enacted in Korea in 2015.

Design

Longitudinal observational study of the interests of healthcare stakeholders generated between January 2014 and September 2018.

Materials and methods

Text data were collected from 2,487 documents on 18 websites that were identified as representative healthcare stakeholder groups of consumers, providers, government, and researchers. A Korean natural language processing (NLP) package, manual review, and synonym dictionary were used for data preprocessing, and we adopted the unsupervised NLP method of probabilistic topic modeling and latent Dirichlet allocation. A linear trend analysis over time, a qualitative step involving two external experts, and original text reviews were performed to validate the identified topics.

Results

Forty-one topics were identified, and the most common concerns of stakeholders were institutional infection control as triggered by the Middle East respiratory syndrome outbreak in early 2015, and infusion-related infection from late 2017 until the middle of 2018. The other top-three concerns of the stakeholder groups were highly similar, while research topics were limited to the perceptions of providers and the activities and culture of hospitals. Five topics showed statistically significant increasing trends over time, while another five showed decreasing trends (both P < 0.05). In the qualitative step, we confirmed 35 themes and revised the other 6.

Conclusions

A common concern among stakeholders was hospital infection control, ranging from nosocomial infections to those brought in by family visiting patients. Government policies and systemic approaches to patient safety were highlighted by different stakeholders. Researchers were focused on hospital sociocultural factors at both the organizational and clinician levels. These identified concerns all should be advocated by the public health policy.

Ambulatory care-sensitive emergency department cases: a mixed methods approach to systemize and analyze cases in Germany

BACKGROUND

Internationally, emergency departments (ED) are treating increasing numbers of patients with conditions that could have been managed appropriately in ambulatory care (AC) settings. The aim of our study was to develop the first consensus-based list of AC-sensitive conditions commonly seen in German EDs and explore predictors of these visits.

METHODS

Our study used a Delphi survey of 30 physicians to compile a list of conditions they agreed were amenable to AC treatment. The group identified reasons why patients visit EDs instead of AC. We used the results to inform spatial regression models analysing the association of patient characteristics and attributes of AC with AC-sensitive ED visits based on 2015 district-level data.

RESULTS

Our study provides a list of AC-sensitive conditions based on the German ED context. Results suggest that, up to the age of 70 years, the older the patients, the less likely they seek EDs for these conditions. Results of our regression analyses suggest that AC-sensitive ED rates were significantly higher in districts with lower physician density. Patients' urgency perception and preferences were identified as main drivers of AC-sensitive ED visits.

CONCLUSION

Future policy measures should aim to help guide patients through the healthcare system so that they receive the best care in place that is most appropriate in terms of quality, safety and continuity of care. A list of AC-sensitive ED conditions can be used as a monitoring instrument and for further analyses of routine data to inform policy makers seeking to improve resource use and allocation.

Key elements for the education and counselling of patients treated with oral anticancer drugs

PURPOSE

The process of education and counselling of patients treated with oral anticancer drugs, aimed to achieve adequate self-management, involves different stakeholders (i.e. physicians, oncology nurses, pharmacists) from primary and secondary care. However, currently no guiding principles exist on how to organize and perform education and counselling for these patients. Therefore, the purpose of this study was to develop and build consensus on key elements for the education and counselling in patients on oral anticancer drugs.

METHODS

A multi-method approach combining a literature search, semi-structured interviews with patients and healthcare professionals, and input from experts were used to develop an initial list of key elements. Subsequently, consensus was built in a two-round Delphi-study, involving patients and healthcare professionals from primary and secondary care. Key elements were retained if at least 80% of respondents in all groups considered the element as important or if in at least one group 90% consensus was reached.

RESULTS

The initial list contained 110 key elements, divided in 7 themes: coordination of care, patient contacts: style and content, medication counselling at the start of the treatment and during follow-up, psychosocial support, and involvement of family and friends. After the first Delphi round, 80% consensus was reached for 23 elements; 6 new key elements were added. After the second round, 80% consensus was reached for 51 out of 116 elements; 31 elements were added following the 90%-rule, yielding a list of 82 elements.

CONCLUSION

The final list of 82 key elements, obtained in this study, could be used to develop clinical pathways that guide adequate education and counselling of patients on oral anticancer drugs. Due to the open description, the implementation of these elements can be adapted to the specific context and composition of the oncology team.

Clinical indicators for common paediatric conditions: Processes, provenance and products of the CareTrack Kids study

BACKGROUND

In order to determine the extent to which care delivered to children is appropriate (in line with evidence-based care and/or clinical practice guidelines (CPGs)) in Australia, we developed a set of clinical indicators for 21 common paediatric medical conditions for use across a range of primary, secondary and tertiary healthcare practice facilities.

METHODS

Clinical indicators were extracted from recommendations found through systematic searches of national and international guidelines, and formatted with explicit criteria for inclusion, exclusion, time frame and setting. Experts reviewed the indicators using a multi-round modified Delphi process and collaborative online wiki to develop consensus on what constituted appropriate care.

RESULTS

From 121 clinical practice guidelines, 1098 recommendations were used to draft 451 proposed appropriateness indicators. In total, 61 experts (n = 24 internal reviewers, n = 37 external reviewers) reviewed these indicators over 40 weeks. A final set of 234 indicators resulted, from which 597 indicator items were derived suitable for medical record audit. Most indicator items were geared towards capturing information about under-use in healthcare (n = 551, 92%) across emergency department (n = 457, 77%), hospital (n = 450, 75%) and general practice (n = 434, 73%) healthcare facilities, and based on consensus level recommendations (n = 451, 76%). The main reason for rejecting indicators was 'feasibility' (likely to be able to be used for determining compliance with 'appropriate care' from medical record audit).

CONCLUSION

A set of indicators was developed for the appropriateness of care for 21 paediatric conditions. We describe the processes (methods), provenance (origins and evolution of indicators) and products (indicator characteristics) of creating clinical indicators within the context of Australian healthcare settings. Developing consensus on clinical appropriateness indicators using a Delphi approach and collaborative online wiki has methodological utility. The final indicator set can be used by clinicians and organisations to measure and reflect on their own practice.

Neurocognitive disorders: what are the prioritized caregiver needs? A consensus obtained by the Delphi method

Background

The symptoms related to neurocognitive disorders (NCD) may lead to caregiver burden increase. Involving caregivers in research may be an effective way of improving the practicalities and relevance of interventions. The aim of this study was to gather opinion and gain consensus on the caregivers ‘priorities, using a Delphi method and including aspects of needs in pharmaceutical dimension.

Methods

Observational study using a modified Delphi method. This study was conducted in the Clinical and Research Memory Center of the University Hospital of Lyon (France), between September 2015 and January 2016. The expert panel was composed of 68 informal caregivers of people with subjective cognitive decline or NCD living at home.

Results

Caregivers assigned a very high importance to the dimension “information needs about their relative’s disease”, i.e. information on the disease, the treatment and the research; and to “coping skills”, i.e. skills related to emotional support, communication, relationship evolution with the relative and skills to cope with behavioural crisis, behavioural and cognitive disorders. The aspect “coping with behavioural disorders” received a high selection rate (83%).

Conclusions

The main needs selected can be used to design relevant interventions and give guidance to policy to support caregivers. To meet caregiver’s needs, interventions should focus on information about disease and treatment and psychoeducational interventions.

Why Do People Resist Patient Portal Systems?

The implementation of patient portal systems (PPS) has potential benefits to both healthcare providers and their patients. However, evidence shows that PPS are being resisted by patients. Little research in IS has addressed this phenomenon. To understand PPS resistance, this study uses the dual factor model of IT usage to develop an integration of the user resistance model (URM) with the universal theory of acceptance and use of technology (UTAUT). Survey data were used to test the integrative model. A total of 265 responses were gathered from patients at a large international hospital. The data were analyzed using structural equation modeling (EQM). The results revealed resistance to change as an inhibitor to intentions to use the PPS. This study demonstrated the importance of integrating resistance to change with the technology use research, especially in healthcare settings. Moreover, this study is considered to be among the few studies in IS to incorporate patients' perspectives regarding new healthcare technology.

Perceptions of health managers and professionals about mental health and primary care integration in Rio de Janeiro: a mixed methods study

BACKGROUND

Community-based primary mental health care is recommended in low and middle-income countries. The Brazilian Health System has been restructuring primary care by expanding its Family Health Strategy. Due to mental health problems, psychosocial vulnerability and accessibility, Matrix Support teams are being set up to broaden the professional scope of primary care. This paper aims to analyse the perceptions of health professionals and managers about the integration of primary care and mental health.

METHOD

In this mixed-method study 18 health managers and 24 professionals were interviewed from different primary and mental health care services in Rio de Janeiro. A semi-structured survey was conducted with 185 closed questions ranging from 1 to 5 and one open-ended question, to evaluate: access, gateway, trust, family focus, primary mental health interventions, mental health records, mental health problems, team collaboration, integration with community resources and primary mental health education. Two comparisons were made: health managers and professionals' (Mann-Whitney non-parametric test) and health managers' perceptions (Kruskall-Wallis non parametric-test) in 4 service designs (General Traditional Outpatients, Mental Health Specialised Outpatients, Psychosocial Community Centre and Family Health Strategy)(SPSS version 17.0). Qualitative data were subjected to Framework Analysis.

RESULTS

Firstly, health managers and professionals' perceptions converged in all components, except the health record system. Secondly, managers' perceptions in traditional services contrasted with managers' perceptions in community-based services in components such as mental health interventions and team collaboration, and converged in gateway, trust, record system and primary mental health education. Qualitative data revealed an acceptance of mental health and primary care integration, but a lack of communication between institutions. The Mixed Method demonstrated that interviewees consider mental health and primary care integration as a requirement of the system, while their perceptions and the model of work produced by the institutional culture are inextricably linked.

CONCLUSION

There is a gap between health managers' and professionals' understanding of community-based primary mental health care. The integration of different processes of work entails both rethinking workforce actions and institutional support to help make changes.

Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development

BACKGROUND

Methods for developing a core outcome or information set require involvement of key stakeholders to prioritise many items and achieve agreement as to the core set. The Delphi technique requires participants to rate the importance of items in sequential questionnaires (or rounds) with feedback provided in each subsequent round such that participants are able to consider the views of others. This study examines the impact of receiving feedback from different stakeholder groups, on the subsequent rating of items and the level of agreement between stakeholders.

METHODS

Randomized controlled trials were nested within the development of three core sets each including a Delphi process with two rounds of questionnaires, completed by patients and health professionals. Participants rated items from 1 (not essential) to 9 (absolutely essential). For round 2, participants were randomized to receive feedback from their peer stakeholder group only (peer) or both stakeholder groups separately (multiple). Decisions as to which items to retain following each round were determined by pre-specified criteria.

RESULTS

Whilst type of feedback did not impact on the percentage of items for which a participant subsequently changed their rating, or the magnitude of change, it did impact on items retained at the end of round 2. Each core set contained discordant items retained by one feedback group but not the other (3-22 % discordant items). Consensus between patients and professionals in items to retain was greater amongst those receiving multiple group feedback in each core set (65-82 % agreement for peer-only feedback versus 74-94 % for multiple feedback). In addition, differences in round 2 scores were smaller between stakeholder groups receiving multiple feedback than between those receiving peer group feedback only. Variability in item scores across stakeholders was reduced following any feedback but this reduction was consistently greater amongst the multiple feedback group.

CONCLUSIONS

In the development of a core outcome or information set, providing feedback within Delphi questionnaires from all stakeholder groups separately may influence the final core set and improve consensus between the groups. Further work is needed to better understand how participants rate and re-rate items within a Delphi process.

TRIAL REGISTRATION

The three randomized controlled trials reported here were each nested within the development of a core information or outcome set to investigate processes in core outcome and information set development. Outcomes were not health-related and therefore trial registration was not applicable.

Agreeing the content of a patient-reported outcome measure for primary care: a Delphi consensus study

BACKGROUND

As the first contact for any health-related need, primary care clinicians often address multiple patient problems, with a range of possible outcomes. There is currently no patient-reported outcome measure (PROM) which covers this range of outcomes. Therefore, many research studies into primary care services use PROMs that do not capture the full impact of these services.

OBJECTIVE

The study aim was to identify outcomes sought by primary care patients which clinicians can influence, thus providing the basis for a new primary care PROM.

METHODS

We used a Delphi process starting with an outcomes list inductively derived in a prior qualitative study. Thirty-five experts were recruited into patient, clinician and academic panels. Participants rated each outcome on whether it was (i) relevant to health, (ii) influenced by primary care and (iii) detectable by patients. In each round, outcomes which passed/failed preset levels of agreement were accepted/rejected. Remaining outcomes continued to the next round.

RESULTS

The process resulted in a set of outcomes occupying the domains of health status, health empowerment (internal and external), and health perceptions. Twenty-six of 36 outcomes were accepted for inclusion in a PROM. Primary care having insufficient influence was the main reason for exclusion.

CONCLUSIONS

To our knowledge, this is the first time PROM outcomes have been agreed through criteria which explicitly exclude outcomes less relevant to health, uninfluenced by primary care or undetected by patients. The PROM in development covers a unique set of outcomes and offers an opportunity for enhanced research into primary care.

Initial development of patient-reported instrument assessing harm, efficacy, and misuse of long-term opioid therapy

Guidelines on long-term opioid therapy recommend frequent reassessment of harm, efficacy, and misuse of these potentially harmful and sometimes ineffective medications. In primary care, there is a need for a brief, patient-reported instrument. This report details the initial steps in the development of such an instrument. An interdisciplinary team of clinician-scientists performed four discrete steps in this study: (1) conceptualization of the purpose and function of the instrument, (2) assembly of an item pool, (3) expert rating on which items were most important to include in the instrument, and (4) modification of expert-selected items based on a reading level check and cognitive interviews with patients. A diverse panel of 47 subject matter experts was presented with 69 items to rate on a 1-9 scale in terms of importance for inclusion in the instrument. The panel highly rated 37 items: 8 related to harm, 4 related to efficacy, and 25 related to misuse. These 37 items were then tested for patient comprehension and modified as needed. Next steps in development will include further item reduction, testing against a gold standard, and assessment of the instrument's effect on clinical outcomes.

User-generated quality standards for youth mental health in primary care: a participatory research design using mixed methods

OBJECTIVES

To develop user-generated quality standards for young people with mental health problems in primary care using a participatory research model.

METHODS

50 young people aged 16-25 from community settings and primary care participated in focus groups and interviews about their views and experiences of seeking help for mental health problems in primary care, cofacilitated by young service users and repeated to ensure respondent validation. A second group of young people also aged 16-25 who had sought help for any mental health problem from primary care or secondary care within the last 5 years were trained as focus groups cofacilitators (n=12) developed the quality standards from the qualitative data and participated in four nominal groups (n=28).

RESULTS

46 quality standards were developed and ranked by young service users. Agreement was defined as 100% of scores within a two-point region. Group consensus existed for 16 quality standards representing the following aspects of primary care: better advertising and information (three); improved competence through mental health training and skill mix within the practice (two); alternatives to medication (three); improved referral protocol (three); and specific questions and reassurances (five). Alternatives to medication and specific questions and reassurances are aspects of quality which have not been previously reported.

CONCLUSIONS

We have demonstrated the feasibility of using participatory research methods in order to develop user-generated quality standards. The development of patient-generated quality standards may offer a more formal method of incorporating the views of service users into quality improvement initiatives. This method can be adapted for generating quality standards applicable to other patient groups.

Development of Algo, a clinical algorithm for non–occupational therapists selecting bathing equipment

Background.

In Quebec, occupational therapy guidelines allow non–occupational therapists, such as home health aides, to select bathing equipment for “straightforward” cases of clients living at home as long as the aides use a decision-making tool.

Purpose.

Our aim was to develop a tool that met the common needs of Quebec’s health and social services centres (HSSCs), which involve home health aides in selecting bathing equipment for home-dwelling clients.

Method.

We followed an ongoing iterative process involving a literature review as well as (a) a synthesis of 40 in-house tools, (b) feedback from 10 occupational therapists (two questionnaires and one focus group), (c) pretests, and (d) translation.

Findings.

Algo is a clinical algorithm constituting a visual map of the logical steps to follow when selecting bathing equipment for straightforward cases. Algo is a series of yes/no questions dealing with occupation, person, and environment.

Implications.

Algo, rooted in evidence and regulatory board guidelines, is available to HSSCs involving non-occupational therapists in selecting bathing equipment.

Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers

PURPOSE

Simple, valid, and evidence-based indicators to measure the quality of antimicrobial prescribing in acute-care hospitals are urgently needed and increasingly requested by policymakers. The aim of this study was to develop new consensus quality indicators (QIs) for hospital antibiotic stewardship (ABS) and infection management which will be further evaluated for internal quality management and external quality assessment in Germany.

METHODS

Based on an extensive literature review, the Austrian-German hospital ABS Guideline Committee and selected members of the German ABS Expert Network discussed and drafted a list of 99 potential indicators for hospitals that reflect structural prerequisites for ABS (35 items), ABS core activities (18 items), additional ABS measures (5 items), and process of care indicators (both generic and disease-specific-12 and 29 items, respectively). Questionnaires were mailed to German ABS experts and healthcare professionals with further education in ABS. Participants scored (on a nine-point Likert scale) relevance (clinical, ecological/resistance, economical/expenses) and presumed practicability (six categories: clarity of definition, effort to collect data, barrier to implementation, verifiability, suitability for external quality assessment, quality gap), taking into account their local work environment. The scores were processed according to the RAND/UCLA appropriateness method, and QIs were judged relevant if the median (clinical + ecological and/or economical) scores were >6. The indicators thus assessed to be potentially relevant were then filtered according to their practicability. Highly relevant QIs with borderline practicability scores and items with disagreements and overlapping areas were re-discussed in a final multidisciplinary panel consensus workshop convened in November 2012.

RESULTS

Of the 340 questionnaires that were mailed, 75 questionnaires were completed and returned. Of 99 initially proposed items, 32 were excluded due to insufficient scores. Of the remaining 67 items, 21 structural and 21 process of care QIs were finally selected, including four QIs with high clinical and ecological but limited economical relevance, and three QIs with high clinical and economical but limited ecological relevance. Among the selected QIs, efforts to collect data and implementation barriers were scored as suboptimal in many cases.

CONCLUSIONS

A catalog of consensus structural and process of care ABS-QIs was established. These should undergo further pilot and feasibility studies in the German hospital healthcare sector. The panelists were most critical regarding resource use/complexity issues and presumed implementation barriers. How this may limit applicability of QIs remains to be determined.

Clinical stakeholders' opinions on the use of selective decontamination of the digestive tract in critically ill patients in intensive care units: an international Delphi study

Introduction

Selective decontamination of the digestive tract (SDD) is a prophylactic antibiotic regimen that is not widely used in practice. We aimed to describe the opinions of key ‘stakeholders’ about the validity of the existing evidence base, likely consequences of implementation, relative importance of their opinions in influencing overall practice, likely barriers to implementation and perceptions of the requirement for further research to inform the decision about whether to embark on a further large randomised controlled trial.

Methods

This was a Delphi study informed by comprehensive framework of possible determinants of health professionals’ behaviour to study Critical Care practice in four countries. There were four key stakeholder participant groups including ICU physicians, pharmacists, clinical leads, and clinical microbiologists/ infectious disease physicians. Round one comprised participant interviews and Rounds two and three were online questionnaires using Delphi method.

Results

In this study, 141 participants were recruited of whom 82% were retained. Participants rated themselves as knowledgeable about SDD. Antibiotic resistance was identified as the most important issue. SDD was seen as a low clinical priority but few participants reported strong opposition. There was moderate agreement that research to date has not adequately addressed concerns about antibiotic resistance and lacks generalizability. Participants indicated equipoise with regard to benefits and harms of SDD, and indicated strong support for a further randomised trial.

Conclusions

Clinicians have clinical equipoise about the effectiveness of SDD. Future research requires longer follow up to assess antibiotic resistance as well as greater validity/generalizability to provide definitive answers on the effectiveness of decontamination and effects on antibiotic resistance. SDD was regarded as not being a high clinical priority, which may limit future trial participation. These results have identified that further large randomised controlled trial of SDD in critical care is both warranted and appropriate.

Evaluating the quality of colorectal cancer care across the interface of healthcare sectors

Background

Colorectal cancer (CRC) has a high prevalence in western countries. Diagnosis and treatment of CRC is complex and requires multidisciplinary collaboration across the interface of health care sectors. In Germany, a new nationwide established program aims to provide quality information of healthcare delivery across different sectors. Within this context, this study describes the development of a set of quality indicators charting the whole pathway of CRC-care including data specifications that are necessary to operationalize these indicators before practice testing.

Methods

Indicators were developed following a systematic 10 step modified ‘RAND/UCLA Appropriateness Method’ which involved a multidisciplinary panel of thirteen participants. For each indicator in the final set, data specifications relating to sources of quality information, data collection procedures, analysis and feedback were described.

Results

The final indicator set included 52 indicators covering diagnostic procedures (11 indicators), therapeutic management (28 indicators) and follow-up (6 indicators). In addition, 7 indicators represented patient perspectives. Primary surgical tumor resection and pre-operative radiation (rectum carcinoma only) were perceived as most useful tracer procedures initiating quality data collection. To assess the quality of CRC care across sectors, various data sources were identified: medical records, administrative inpatient and outpatient data, sickness-funds billing code systems and patient survey.

Conclusion

In Germany, a set of 52 quality indicators, covering necessary aspects across the interfaces and pathways relevant to CRC-care has been developed. Combining different sectors and sources of health care in quality assessment is an innovative and challenging approach but reflects better the reality of the patient pathway and experience of CRC-care.

Quality indicators for all dimensions of infertility care quality: consensus between professionals and patients

STUDY QUESTION

What is the relative importance of the six dimensions of quality of care according to different stakeholders and can a quality indicator set address all six quality dimensions and incorporate the views from professionals working in different disciplines and from patients?

SUMMARY ANSWER

Safety, effectiveness and patient centeredness were the most important quality dimensions. All six quality dimensions can be assessed with a set of 24 quality indicators, which is face valid and acceptable according to both professionals from different disciplines and patients.

WHAT IS KNOWN ALREADY

To our knowledge, no study has weighted the relative importance of all quality dimensions to infertility care. Additionally, there are very few infertility care-specific quality indicators and no quality indicator set covers all six quality dimensions and incorporated the views of professionals and patients.

STUDY DESIGN, SIZE AND DURATION

A three-round iterative Delphi survey including patients and professionals from four different fields, conducted in two European countries over the course of 2011 and 2012.

PARTICIPANTS/MATERIALS, SETTINGS AND METHODS

Dutch and Belgian gynaecologists, embryologists, counsellors, nurses/midwifes and patients took part (n = 43 in round 1 and finally 30 in round 3). Respondents ranked the six quality dimensions twice for importance and their agreement was evaluated. Furthermore, in round 1, respondents gave suggestions, which were subsequently uniformly formulated as quality indicators. In rounds 2 and 3, respondents rated the quality indicators for preparedness to measure and for importance (relation to quality and prioritization for benchmarking). Providing feedback allowed selecting indicators based on consensus between stakeholder groups. Measurable indicators, important to all stakeholder groups, were selected for each quality dimension.

MAIN RESULTS

All stakeholder groups and most individuals agreed that safety, effectiveness and patient centeredness were the most important quality dimensions. A total of 498 suggestions led to the development of 298 indicators. Professionals were sufficiently prepared to measure 204 of these indicators. Based on importance, 52 (7-15 per dimension; round 2) and finally 24 (4 per dimension; round 3) quality indicators were selected.

LIMITATIONS, REASONS FOR CAUTION

The final quality indicator set does not cover the entire care process, but rather takes a 'sample' of each quality dimension. Although the quality indicators are face valid and acceptable, their psychometric characteristics need to be tested by further research.

WIDER IMPLICATIONS OF THE FINDINGS

Quality management should focus on safety, effectiveness and patient centeredness of care. Clinics can use the quality indicator set to assess all quality dimensions of their care.

Shared mental models of integrated care: aligning multiple stakeholder perspectives

PURPOSE

Health service organizations and professionals are under increasing pressure to work together to deliver integrated patient care. A common understanding of integration strategies may facilitate the delivery of integrated care across inter-organizational and inter-professional boundaries. This paper aims to build a framework for exploring and potentially aligning multiple stakeholder perspectives of systems integration.

DESIGN/METHODOLOGY/APPROACH

The authors draw from the literature on shared mental models, strategic management and change, framing, stakeholder management, and systems theory to develop a new construct, Mental Models of Integrated Care (MMIC), which consists of three types of mental models, i.e. integration-task, system-role, and integration-belief.

FINDINGS

The MMIC construct encompasses many of the known barriers and enablers to integrating care while also providing a comprehensive, theory-based framework of psychological factors that may influence inter-organizational and inter-professional relations. While the existing literature on integration focuses on optimizing structures and processes, the MMIC construct emphasizes the convergence and divergence of stakeholders' knowledge and beliefs, and how these underlying cognitions influence interactions (or lack thereof) across the continuum of care.

PRACTICAL IMPLICATIONS

MMIC may help to: explain what differentiates effective from ineffective integration initiatives; determine system readiness to integrate; diagnose integration problems; and develop interventions for enhancing integrative processes and ultimately the delivery of integrated care.

ORIGINALITY/VALUE

Global interest and ongoing challenges in integrating care underline the need for research on the mental models that characterize the behaviors of actors within health systems; the proposed framework offers a starting point for applying a cognitive perspective to health systems integration.

Simultaneous development of guidelines and quality indicators -- how do guideline groups act? A worldwide survey

PURPOSE

The purpose of this paper is to examine the question of how official bodies, health care organisations, and professional associations deal with the absence of a methodological gold standard for the simultaneous development of clinical practice guidelines and quality indicators, what procedures they use and what they feel are major strengths and limitations of their methods.

DESIGN/METHODOLOGY/APPROACH

The authors conducted a web-based survey among 90 organisational members of the Guidelines International Network (G-I-N) representing 34 countries from Africa, America, Asia, Europe and Oceania. All organisational G-I-N members were invited to participate in the survey by following a link provided in the invitation e-mail.

FINDINGS

The responses of 24 organisations were included in the final analysis. The results indicate a broad variability in the approaches and methods used to develop quality indicators and guidelines simultaneously. The answers of the participants indicated a lack of formal procedures for the simultaneous development. Formal procedures exist in only about half of the participating organisations. In addition, piloting or evaluation of the procedures is almost completely missing. Significantly, respondents mainly reported that the procedure used in their organisation "could certainly be more rigorous". Besides various strengths, participants reported a considerable number of limitations of the development processes they use.

ORIGINALITY/VALUE

This survey among G-I-N members -- despite limitations -- gives helpful insights in the state of the simultaneous development of quality indicators and clinical practice guidelines and underlines the need for future activities in methodological standard development and quality improvement of these processes.

Identifying Characteristics of ‘Straightforward Cases’ for which Support Personnel Could Recommend Home Bathing Equipment

Introduction:

The general consensus is that an occupational therapist should carry out the assessment for recommending home bathing equipment. Nevertheless, in response to a shortage of human resources, home-care occupational therapists in Quebec, Canada, frequently transfer the clinical task of recommending bathing equipment to support personnel in ‘straightforward cases’. However, there is no consensus on what constitutes such cases, and clinicians lack information on how to define a straightforward case.

Objective:

To characterise ‘straightforward cases’ when recommending bathing equipment in home-care occupational therapy.

Design:

The RAND/UCLA Appropriateness Method, combining a literature review with a three-round survey and one focus group meeting of nine occupational therapists.

Results:

Eight characteristics required for describing straightforward cases for bathing equipment recommendations were identified. They cover the three dimensions of the Canadian Model of Occupational Performance and Engagement: the occupation, the person, and the person's home environment.

Conclusion:

The literature review and collective opinion of experienced occupational therapists made it possible to agree on a common language to describe straightforward cases for bathing equipment. The characteristics identified will, it is hoped, support the critical thinking of clinicians deciding whether or not to transfer the task of recommending bathing equipment to support personnel.

Guideline-based development of quality indicators for hypertensive diseases in pregnancy

OBJECTIVE

Hypertensive disorders in pregnancy are one of the main causes of maternal morbidity and mortality. Internationally, several organizations have developed clinical guidelines to assist professionals and to supply patients with the best possible care. To improve the care for this group of patients, insight into the application of clinical guidelines in daily practise is needed. Valid quality indicators are necessary to estimate actual guideline adherence. In this study, we developed a set of valid guideline-based quality indicators for hypertensive diseases in pregnancy.

METHODS

A systematic RAND-modified Delphi method was used to develop a set of quality indicators on the basis of evidence-based guidelines and literature on hypertension and pregnancy. Experts' opinions were used to select the indicators regarding specific criteria such as efficacy, level of health gain, and potential for care improvement.

RESULTS

A representative set of 14 quality indicators was selected from 48 initial guideline recommendations. Indicators concerned both professional performance and organization of care for patients with hypertension in pregnancy.

CONCLUSIONS

This study describes the systematic, stepwise method used to develop quality indicators that can be used to monitor care for hypertensive diseases in pregnancy.

Connecting the voices of users, caregivers and providers on service quality: a study of home-care services

PURPOSE

This article aims to discuss the relevancy of different instruments used to gather information on homecare service quality from multiple stakeholders and the challenges encountered when trying to blend their views for prioritizing areas needing improvement.

DESIGN/METHODOLOGY/APPROACH

The study centers on four homecare agencies: one public, one private for-profit and two not-for-profit services, implementing continuous quality improvement (CQI) programs. Various instruments were tested with random and convenience elderly service user, family caregiver and front-line worker samples. Instrument evaluation included operational effectiveness and agency manageability.

FINDINGS

A qualitative approach, centered on small stakeholder samples, is fairly effective at assessing service quality, yet demands a strong commitment from agencies in personnel time and resources, as well as the necessary skills. Small-size, private homecare providers seem less-well equipped to handle comprehensive assessments without external support More importantly, assessments have to be done strategically, such that timing and work needed does not undermine program viability.

PRACTICAL IMPLICATIONS

The approach and instruments tested have practical implications for decision makers and homecare organization managers interested in CQI.

ORIGINALITY/VALUE

The article systematically evaluates quality assessment and priority-setting instruments applied to various stakeholders and homecare settings.

Quality standards for child and adolescent mental health in primary care

BACKGROUND

Child and adolescent mental health problems are common in primary healthcare settings. However, few parents of children with mental health problems express concerns about these problems during consultations. Based on parental views, we aimed to create quality of care measures for child and adolescent mental health in primary care and develop consensus about the importance of these quality standards within primary care.

METHODS

Quality Standards were developed using an iterative approach involving four phases: 1) 34 parents with concerns about their child's emotional health or behaviour were recruited from a range of community settings including primary care practices to participate in focus group discussions, followed by validation groups or interviews. 2) Preliminary Quality Standards were generated that fully represented the parents' experiences and were refined following feedback from an expert parent nominal group. 3) 55 experts, including parents and representatives from voluntary organisations, across five panels participated in a modified two-stage Delphi study to develop consensus on the importance of the Quality Standards. The panels comprised general practitioners, other community-based professionals, child and adolescent psychiatrists, other child and adolescent mental health professionals and public health and policy specialists. 4) The final set of Quality Standards was piloted with 52 parents in primary care.

RESULTS

In the Delphi process, all five panels agreed that 10 of 31 Quality Standards were important. Although four panels rated 25-27 statements as important, the general practitioner panel rated 12 as important. The final 10 Quality Standards reflected healthcare domains involving access, confidentiality for young people, practitioner knowledge, communication, continuity of care, and referral to other services. Parents in primary care agreed that all 10 statements were important.

CONCLUSIONS

It is feasible to develop a set of Quality Standards to assess mental healthcare provision for children and adolescents seen within primary healthcare services. Primary care practitioners should be aware of parental perspectives about quality of care as these may influence help-seeking behaviours.

Assessment and treatment of malnutrition in Dutch geriatric practice: consensus through a modified Delphi study

OBJECTIVE

scientific evidence regarding the optimal management of malnutrition in geriatric patients is scarce. Our aim was to develop a consensus statement for geriatric hospital practice concerning six elements: (i) definition of malnutrition, (ii) screening and assessment, (iii) treatment and monitoring, (iv) roles and responsibilities of involved health care professionals, (v) communication and coordination of care between hospital and community health care professionals, (vi) quality indicators for malnutrition management.

DESIGN

a modified Delphi study.

METHODS

eleven geriatricians with special interest in malnutrition participated. In four rounds the experts rated the relevance of 204 statements, which were based on a literature review, on a five-point Likert scale. From the responses, means and 95% CIs were calculated. Consensus was defined as a lower 95% confidence limit ≥4.0.

RESULTS

the panel reached consensus that malnutrition should be considered a geriatric syndrome. The nutritional status should be assessed using the Mini Nutritional Assessment combined with comprehensive geriatric assessment. Nutritional interventions should be combined with interventions targeting underlying factors. Specific goals for nutritional therapy and ways to achieve them were agreed upon. According to the experts, malnutrition is best managed by a multidisciplinary team for whom roles and responsibilities were specified. At discharge written information about the nutritional problem, treatment plan and goals should be provided to the patient, caregiver and community health care professionals.

CONCLUSION

this study shows that a qualitative study based on a modified Delphi technique can result in national consensus on essential ingredients for a practical malnutrition guideline for geriatric patients.

Methods for the guideline-based development of quality indicators--a systematic review

BACKGROUND

Quality indicators (QIs) are used in many healthcare settings to measure, compare, and improve quality of care. For the efficient development of high-quality QIs, rigorous, approved, and evidence-based development methods are needed. Clinical practice guidelines are a suitable source to derive QIs from, but no gold standard for guideline-based QI development exists. This review aims to identify, describe, and compare methodological approaches to guideline-based QI development.

METHODS

We systematically searched medical literature databases (Medline, EMBASE, and CINAHL) and grey literature. Two researchers selected publications reporting methodological approaches to guideline-based QI development. In order to describe and compare methodological approaches used in these publications, we extracted detailed information on common steps of guideline-based QI development (topic selection, guideline selection, extraction of recommendations, QI selection, practice test, and implementation) to predesigned extraction tables.

RESULTS

From 8,697 hits in the database search and several grey literature documents, we selected 48 relevant references. The studies were of heterogeneous type and quality. We found no randomized controlled trial or other studies comparing the ability of different methodological approaches to guideline-based development to generate high-quality QIs. The relevant publications featured a wide variety of methodological approaches to guideline-based QI development, especially regarding guideline selection and extraction of recommendations. Only a few studies reported patient involvement.

CONCLUSIONS

Further research is needed to determine which elements of the methodological approaches identified, described, and compared in this review are best suited to constitute a gold standard for guideline-based QI development. For this research, we provide a comprehensive groundwork.

Conducting online expert panels: a feasibility and experimental replicability study

Background

This paper has two goals. First, we explore the feasibility of conducting online expert panels to facilitate consensus finding among a large number of geographically distributed stakeholders. Second, we test the replicability of panel findings across four panels of different size.

Method

We engaged 119 panelists in an iterative process to identify definitional features of Continuous Quality Improvement (CQI). We conducted four parallel online panels of different size through three one-week phases by using the RAND's ExpertLens process. In Phase I, participants rated potentially definitional CQI features. In Phase II, they discussed rating results online, using asynchronous, anonymous discussion boards. In Phase III, panelists re-rated Phase I features and reported on their experiences as participants.

Results

66% of invited experts participated in all three phases. 62% of Phase I participants contributed to Phase II discussions and 87% of them completed Phase III. Panel disagreement, measured by the mean absolute deviation from the median (MAD-M), decreased after group feedback and discussion in 36 out of 43 judgments about CQI features. Agreement between the four panels after Phase III was fair (four-way kappa = 0.36); they agreed on the status of five out of eleven CQI features. Results of the post-completion survey suggest that participants were generally satisfied with the online process. Compared to participants in smaller panels, those in larger panels were more likely to agree that they had debated each others' view points.

Conclusion

It is feasible to conduct online expert panels intended to facilitate consensus finding among geographically distributed participants. The online approach may be practical for engaging large and diverse groups of stakeholders around a range of health services research topics and can help conduct multiple parallel panels to test for the reproducibility of panel conclusions.

Developing an interview guide to evaluate practice-based evidence in nutrition: use of the delphi technique

An interview guide was created for qualitative evaluation of the impact of Practice-based Evidence in Nutrition (PEN) on dietetic practice, and its success as a medium for knowledge translation and transfer (KTT). The Delphi technique was used to bring together a diverse group of experts (n=7) with extensive knowledge in KTT and evidence-based practice (EBP); these experts developed the interview guide content. The technique is an effective means of gathering expert input to inform evaluation tool development, particularly in the absence of accepted evaluation guidelines or pre-existing evaluation tools. Although challenges exist with the Delphi technique, it is an adaptable method that can be modified to meet a variety of needs. During this project, the technique was modified to meet specific needs, including participants' partial anonymity and starting material to reduce the number of required rounds. The resulting interview guide contained open-ended questions focused on respondents' understanding of EBP and PEN, use of PEN by dietitians in other disciplines, perceptions of the quality/usefulness of PEN, and barriers to and facilitators of PEN use.

Development of guidelines for caregivers of people with bipolar disorder: a Delphi expert consensus study

OBJECTIVES

  Close family and friends are often a primary source of support for a person with bipolar disorder. However, there is a lack of information for caregivers about ways to provide helpful support and take care of themselves. Rates of caregiver burden are high and increase the risk of caregiver depression and health problems. This study aimed to develop guidelines to assist caregivers of adults with bipolar disorder to be informed about bipolar disorder and to support the person without neglecting their own wellbeing.

METHODS

  The Delphi method was used to assess consensus between international expert panels of 45 caregivers, 47 consumers, and 51 clinicians about what information to include in the caregiver guidelines. Initial online survey items were based on the existing literature. Subsequent surveys included new or reworded items suggested by panel members and items that needed re-rating. Items endorsed by at least 80% of all three panels formed the content of the guidelines.

RESULTS

  Nearly 86% of the 626 survey items were endorsed. The items covered information on the illness, treatment, and suggestions on ways caregivers can provide support and take care of themselves in the different phases of illness and wellness, and information on dealing with specific real-life challenges. Although consensus rates were high, meaningful areas of difference between panels were found (e.g., collaboration issues).

CONCLUSIONS

  The guidelines provide comprehensive introductory information, suggestions, and resources for caregivers. Access to relevant information may help caregivers to cope constructively with the person's bipolar disorder and their caregiving situation. The content of the guidelines could be used to help formulate a stepped-care approach to supporting caregivers, ranging from basic information and pamphlets to brief training courses and specialized family or caregiver interventions based on need and accessibility.

Assessment of a novel hybrid Delphi and Nominal Groups technique to evaluate quality indicators

OBJECTIVE

To test the implementation of a novel structured panel process in the evaluation of quality indicators.

DATA SOURCE

National panel of 64 clinicians rating usefulness of indicator applications in 2008-2009.

STUDY DESIGN

Hybrid panel combined Delphi Group and Nominal Group (NG) techniques to evaluate 81 indicator applications.

PRINCIPAL FINDINGS

The Delphi Group and NG rated 56 percent of indicator applications similarly. Group assignment (Delphi versus Nominal) was not significantly associated with mean ratings, but specialty and research interests of panelists, and indicator factors such as denominator level and proposed use were. Rating distributions narrowed significantly in 20.8 percent of applications between review rounds.

CONCLUSIONS

The hybrid panel process facilitated information exchange and tightened rating distributions. Future assessments of this method might include a control panel.

Do the organizational reforms of general practice care meet users' concerns? The contribution of the Delphi method

CONTEXT

The debate over primary care reform in France, as in most OECD countries, centres on questions about efficacy and accessibility. Do these reforms actually respond to the users' concerns?

OBJECTIVE

The objective of this study was to identify the importance that users attribute to different aspects of general practice (GP) care.

DESIGN

The method used was a variant of the classical Delphi approach, called Delphi 'ranking-type'. Between May and September 2009, 74 experts aged over 18 were recruited by 'snowballing' sampling. Three iterative rounds were required to identify the core aspects through a consensus-building approach.

RESULTS

It is shown that users attribute a very high importance to the 'doctor-patient relationship' dimension. The following aspects 'GP patient information about his/her illness', 'Clarity of communication and explanation', and 'Whether the GP seemed listen to the patient' were evaluated by 96% of the experts as being of high importance. The coordination of GP was also considered as a very important aspect for 85% of the experts. In contrast, the aspects that belong to the organizational dimension appeared to be of relatively low importance for users.

CONCLUSIONS

Our results support a comprehensive approach of care and argue in favour of care reorganization following the patient-centred model. To promote organizational care reforms through the prism of the doctor-patient relationship could thus be a fruitful way to insure a better quality of care and the social acceptability of the reforms.

How to optimize tuberculosis case finding: explorations for Indonesia with a health system model

Background

A mathematical model was designed to explore the impact of three strategies for better tuberculosis case finding. Strategies included: (1) reducing the number of tuberculosis patients who do not seek care; (2) reducing diagnostic delay; and (3) engaging non-DOTS providers in the referral of tuberculosis suspects to DOTS services in the Indonesian health system context. The impact of these strategies on tuberculosis mortality and treatment outcome was estimated using a mathematical model of the Indonesian health system.

Methods

The model consists of multiple compartments representing logical movement of a respiratory symptomatic (tuberculosis suspect) through the health system, including patient- and health system delays. Main outputs of the model are tuberculosis death rate and treatment outcome (i.e. full or partial cure). We quantified the model parameters for the Jogjakarta province context, using a two round Delphi survey with five Indonesian tuberculosis experts.

Results

The model validation shows that four critical model outputs (average duration of symptom onset to treatment, detection rate, cure rate, and death rate) were reasonably close to existing available data, erring towards more optimistic outcomes than are actually reported. The model predicted that an intervention to reduce the proportion of tuberculosis patients who never seek care would have the biggest impact on tuberculosis death prevention, while an intervention resulting in more referrals of tuberculosis suspects to DOTS facilities would yield higher cure rates. This finding is similar for situations where the alternative sector is a more important health resource, such as in most other parts of Indonesia.

Conclusion

We used mathematical modeling to explore the impact of Indonesian health system interventions on tuberculosis treatment outcome and deaths. Because detailed data were not available regarding the current Indonesian population, we relied on expert opinion to quantify the parameters. The fact that the model output showed similar results to epidemiological data suggests that the experts had an accurate understanding of this subject, thereby reassuring the quality of our predictions. The model highlighted the potential effectiveness of active case finding of tuberculosis patients with limited access to DOTS facilities in the developing country setting.

Identifying characteristics of residential care facilities relevant to the placement process of seniors

OBJECTIVE

To identify which characteristics of the physical and organizational environments of residential care facilities (RCFs) need to be known to support the placement process of seniors with cognitive and physical disabilities.

DESIGN

We used a 2-round postal Delphi survey based on the RAND/UCLA Appropriateness Method with 2 groups of experts.

PARTICIPANTS

Participants are representatives of concerned stakeholder groups from various sectors (public, private, and community) and geographical regions within the province of Quebec, Canada.

RESULTS

Of the 58 persons who were interested in participating, 48 (82.8%) returned their questionnaires. The first group identified 171 items essential to consider when a cognitively impaired senior needs to move into an RCF. Of these, 88 were related to the physical environment of the RCF and 83 to its organizational environment. The second group identified 146 items to be relevant to consider when a senior with physical impairments needs to choose an RCF; 75 for the physical and 71 for the organizational environment. 85% of the items selected by both groups (cognitive versus physical group) were identical.

CONCLUSION

This study provided a list of important items that described residential care facilities in regard to their physical and organizational environments to support the placement process of seniors. It also provides the possibility to develop a self-administered tool that will supply valuable information to seniors and their relatives, to health care professionals, and to administrators and decisions-makers.

Steps, challenges and lessons in developing community mental health care

This paper summarises our own accumulated experience from developing community-orientated mental health services in England and Italy over the last 20-30 years. From this we have provisionally concluded that the following issues are central to the development of balanced mental health services: a) services need to reflect the priorities of service users and carers; b) evidence supports the need for both hospital and community services; c) services need to be provided close to home; d) some services need to be mobile rather than static; e) interventions need to address both symptoms and disabilities; and f) treatment has to be specific to individual needs. In this paper we consider ten key challenges that often face those trying to develop community-based mental health services: a) dealing with anxiety and uncertainty; b) compensating for a possible lack of structure in community services; c) learning how to initiate new developments; d) managing opposition to change within the mental health system; e) responding to opposition from neighbours; f) negotiating financial obstacles; g) avoiding system rigidities; h) bridging boundaries and barriers; i) maintaining staff morale; and j) creating locally relevant ser- vices rather than seeking "the right answer" from elsewhere.

Development of a Nutrition Counselling Care Map For Dyslipidemia

Purpose: Care maps or clinical pathways for nutrition therapy of dyslipidemia could add to current practice guidelines, by providing templates for feasible and recommended diet counselling processes. A care map was therefore developed by engaging expert and generalist dietitians and external experts from across Canada in a multi-stage consensus process.

Methods: First, a qualitative study was undertaken with a convenience sample of 12 practitioners to identify possible diet care options, using hypothetical client scenarios and cognitive analysis. Second, these care options were rated for five case scenarios considered typical (overweight clients, with or without clinical cardiovascular disease and other comorbidities, potentially motivated to change, consuming high-fat diets, and facing various major barriers to eating behaviour change). The rating was conducted through a survey of participants. Highly appropriate, recommended, and feasible options for counselling were ranked through a two-round modified Delphi process, with teleconference discussions between rounds.

Results: Forty-nine professionals started the consensus process; 39 (80%) completed all aspects. Numerous care processes were appropriate for all clients, with additional focus on barriers for low-income clients, sodium intake for clients with hypertension, and smoking cessation in smokers.

Conclusions: The resulting care map, “Dietitians’ Quick Reference Guide for Clinical Nutrition Therapy for Overweight Clients with Dyslipidemia,” provides a basis for current practice and new effectiveness studies.