A CBPR-Enhanced Delphi Method: The Measurement Approaches to Partnership Success Case Study

As part of a 5-year study to develop and validate an instrument for measuring success in long-standing community-based participatory research (CBPR) partnerships, we utilized the Delphi method with a panel of 16 community and academic CBPR experts to assess face and content validity of the instrument's broad concepts of success and measurement items. In addition to incorporating quantitative and qualitative feedback from two online surveys, we included a 2-day face-to-face meeting with the Expert Panel to invite open discussion and diversity of opinion in line with the CBPR principles framing and guiding the study. The face-to-face meeting allowed experts to review the survey data (with maintained anonymity), convey their perspectives, and offer interpretations that were untapped in the online surveys. Using a CBPR approach facilitated a synergistic process that moved above and beyond the consensus achieved in the initial Delphi rounds, to enhance the Delphi technique and the development of items in the instrument.

Consumers' and health providers' views and perceptions of partnering to improve health services design, delivery and evaluation: a co-produced qualitative evidence synthesis

BACKGROUND

Partnering with consumers in the planning, delivery and evaluation of health services is an essential component of person-centred care. There are many ways to partner with consumers to improve health services, including formal group partnerships (such as committees, boards or steering groups). However, consumers' and health providers' views and experiences of formal group partnerships remain unclear. In this qualitative evidence synthesis (QES), we focus specifically on formal group partnerships where health providers and consumers share decision-making about planning, delivering and/or evaluating health services. Formal group partnerships were selected because they are widely used throughout the world to improve person-centred care. For the purposes of this QES, the term 'consumer' refers to a person who is a patient, carer or community member who brings their perspective to health service partnerships. 'Health provider' refers to a person with a health policy, management, administrative or clinical role who participates in formal partnerships in an advisory or representative capacity. This QES was co-produced with a Stakeholder Panel of consumers and health providers. The QES was undertaken concurrently with a Cochrane intervention review entitled Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation.

OBJECTIVES

1. To synthesise the views and experiences of consumers and health providers of formal partnership approaches that aimed to improve planning, delivery or evaluation of health services. 2. To identify best practice principles for formal partnership approaches in health services by understanding consumers' and health providers' views and experiences.

SEARCH METHODS

We searched MEDLINE, Embase, PsycINFO and CINAHL for studies published between January 2000 and October 2018. We also searched grey literature sources including websites of relevant research and policy organisations involved in promoting person-centred care.

SELECTION CRITERIA

We included qualitative studies that explored consumers' and health providers' perceptions and experiences of partnering in formal group formats to improve the planning, delivery or evaluation of health services.

DATA COLLECTION AND ANALYSIS

Following completion of abstract and full-text screening, we used purposive sampling to select a sample of eligible studies that covered a range of pre-defined criteria, including rich data, range of countries and country income level, settings, participants, and types of partnership activities. A Framework Synthesis approach was used to synthesise the findings of the sample. We appraised the quality of each study using the CASP (Critical Appraisal Skill Program) tool. We assessed our confidence in the findings using the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach. The Stakeholder Panel was involved in each stage of the review from development of the protocol to development of the best practice principles.

MAIN RESULTS

We found 182 studies that were eligible for inclusion. From this group, we selected 33 studies to include in the final synthesis. These studies came from a wide range of countries including 28 from high-income countries and five from low- or middle-income countries (LMICs). Each of the studies included the experiences and views of consumers and/or health providers of partnering in formal group formats. The results were divided into the following categories. Contextual factors influencing partnerships: government policy, policy implementation processes and funding, as well as the organisational context of the health service, could facilitate or impede partnering (moderate level of confidence). Consumer recruitment: consumer recruitment occurred in different ways and consumers managed the recruitment process in a minority of studies only (high level of confidence). Recruiting a range of consumers who were reflective of the clinic's demographic population was considered desirable, particularly by health providers (high level of confidence). Some health providers perceived that individual consumers' experiences were not generalisable to the broader population whereas consumers perceived it could be problematic to aim to represent a broad range of community views (high level of confidence). Partnership dynamics and processes: positive interpersonal dynamics between health providers and consumers facilitated partnerships (high level of confidence). However, formal meeting formats and lack of clarity about the consumer role could constrain consumers' involvement (high level of confidence). Health providers' professional status, technical knowledge and use of jargon were intimidating for some consumers (high level of confidence) and consumers could feel their experiential knowledge was not valued (moderate level of confidence). Consumers could also become frustrated when health providers dominated the meeting agenda (moderate level of confidence) and when they experienced token involvement, such as a lack of decision-making power (high level of confidence) Perceived impacts on partnership participants: partnering could affect health provider and consumer participants in both positive and negative ways (high level of confidence). Perceived impacts on health service planning, delivery and evaluation: partnering was perceived to improve the person-centredness of health service culture (high level of confidence), improve the built environment of the health service (high level of confidence), improve health service design and delivery e.g. facilitate 'out of hours' services or treatment closer to home (high level of confidence), enhance community ownership of health services, particularly in LMICs (moderate level of confidence), and improve consumer involvement in strategic decision-making, under certain conditions (moderate level of confidence). There was limited evidence suggesting partnering may improve health service evaluation (very low level of confidence). Best practice principles for formal partnering to promote person-centred care were developed from these findings. The principles were developed collaboratively with the Stakeholder Panel and included leadership and health service culture; diversity; equity; mutual respect; shared vision and regular communication; shared agendas and decision-making; influence and sustainability.

AUTHORS' CONCLUSIONS

Successful formal group partnerships with consumers require health providers to continually reflect and address power imbalances that may constrain consumers' participation. Such imbalances may be particularly acute in recruitment procedures, meeting structure and content and decision-making processes. Formal group partnerships were perceived to improve the physical environment of health services, the person-centredness of health service culture and health service design and delivery. Implementing the best practice principles may help to address power imbalances, strengthen formal partnering, improve the experiences of consumers and health providers and positively affect partnership outcomes.

Identifying Clinically Relevant Functional Strata to Direct Mobility Preservation among Patients Hospitalized with Medical Conditions

OBJECTIVE

To develop a system to guide interpretation of scores generated from the newly developed item response theory (IRT)-based, multi-dimensional computer adaptive test (FAMCAT) which assesses three important domains of function: mobility, daily activities, and applied cognition.

DESIGN

Cross-sectional data used to inform a modified-Delphi approach to establish FAMCAT cut points delineating various functional ability levels.

SETTING

Large Midwestern academic teaching hospital PARTICIPANTS: 2049 patients hospitalized to an inpatient medical service INTERVENTION: Not applicable.

MAIN OUTCOME MEASURES

FAMCAT Basic Mobility, Daily Activity, and Applied Cognition scores RESULTS: IRT-based score estimation data was successfully integrated with expert clinical feedback using a modified-Delphi process to arrive at consensus yielding 4 functional level strata (ranging from bed-based mobility to independent mobility) for both the FAMCAT Mobility and Daily Activity scales; 1 cut-point was supported to delineate two functional strata for Applied Cognition.

CONCLUSIONS

Meaningful cut-points were established for each FAMCAT domain using a data-informed, modified-Delphi process for achieving consensus. The resulting FAMCAT interpretation guide may be used to develop an ability-matched mobility preservation program and identify patients who may require a higher level of supervision based on the resulting FAMCAT scores.

Consensus minimum hemoglobin level above which patients with myelodysplastic syndromes can safely forgo transfusions

The anemia of MDS often results in decreased quality of life, which is invoked to justify red cell transfusions; however, there are sparse data regarding the minimum hemoglobin (Hb) at which it is safe to forgo transfusions for patients with no evidence of end-organ damage. This issue is even more important in the COVID-19 era, where decreases in blood donations have stressed the blood supply. In March 2018, using a modified Delphi method, we convened a panel of 13 expert MDS clinicians for three iterative rounds to discuss a minimum safe Hb for this population. While the panel was unable to reach the pre-set consensus of 75% for a specific Hb threshold, there was 100% consensus that it be no greater than 7.5 g/dL. Our data suggest that, given no end-organ effects of anemia, patients with MDS can safely forgo transfusions with a Hb of 7.5 g/dL or higher.

Consensus on Quality Indicators of Postgraduate Medical E-Learning: Delphi Study

BACKGROUND

The progressive use of e-learning in postgraduate medical education calls for useful quality indicators. Many evaluation tools exist. However, these are diversely used and their empirical foundation is often lacking.

OBJECTIVE

We aimed to identify an empirically founded set of quality indicators to set the bar for “good enough” e-learning.

METHODS

We performed a Delphi procedure with a group of 13 international education experts and 10 experienced users of e-learning. The questionnaire started with 57 items. These items were the result of a previous literature review and focus group study performed with experts and users. Consensus was met when a rate of agreement of more than two-thirds was achieved.

RESULTS

In the first round, the participants accepted 37 items of the 57 as important, reached no consensus on 20, and added 15 new items. In the second round, we added the comments from the first round to the items on which there was no consensus and added the 15 new items. After this round, a total of 72 items were addressed and, of these, 37 items were accepted and 34 were rejected due to lack of consensus.

CONCLUSIONS

This study produced a list of 37 items that can form the basis of an evaluation tool to evaluate postgraduate medical e-learning. This is, to our knowledge, the first time that quality indicators for postgraduate medical e-learning have been defined and validated. The next step is to create and validate an e-learning evaluation tool from these items.

Application of the RAND-UCLA Appropriateness Methodology to a Large Multidisciplinary Stakeholder Group Evaluating the Validity and Feasibility of Patient-Centered Standards in Geriatric Surgery

OBJECTIVES

To explore (1) differences in validity and feasibility ratings for geriatric surgical standards across a diverse stakeholder group (surgeons vs. nonsurgeons, health care providers vs. nonproviders, including patient-family, advocacy, and regulatory agencies); (2) whether three multidisciplinary discussion subgroups would reach similar conclusions.

DATA SOURCE/STUDY SETTING

Primary data (ratings) were reported from 58 stakeholder organizations.

STUDY DESIGN

An adaptation of the RAND-UCLA Appropriateness Methodology (RAM) process was conducted in May 2016.

DATA COLLECTION/EXTRACTION METHODS

Stakeholders self-administered ratings on paper, returned via mail (Round 1) and in-person (Round 2).

PRINCIPAL FINDINGS

In Round 1, surgeons rated standards more critically (91.2 percent valid; 64.9 percent feasible) than nonsurgeons (100 percent valid; 87.0 percent feasible) but increased ratings in Round 2 (98.7 percent valid; 90.6 percent feasible), aligning with nonsurgeons (99.7 percent valid; 96.1 percent feasible). Three parallel subgroups rated validity at 96.8 percent (group 1), 100 percent (group 2), and 97.4 percent (group 3). Feasibility ratings were 76.9 percent (group 1), 96.1 percent (group 2), and 92.2 percent (group 3).

CONCLUSIONS

There are differences in validity and feasibility ratings by health professions, with surgeons rating standards more critically than nonsurgeons. However, three separate discussion subgroups rated a high proportion (96-100 percent) of standards as valid, indicating the RAM can be successfully applied to a large stakeholder group.