Evaluation in health: participatory methodology and involvement of municipal managers

Consumers' and health providers' views and perceptions of partnering to improve health services design, delivery and evaluation: a co-produced qualitative evidence synthesis

BACKGROUND

Partnering with consumers in the planning, delivery and evaluation of health services is an essential component of person-centred care. There are many ways to partner with consumers to improve health services, including formal group partnerships (such as committees, boards or steering groups). However, consumers' and health providers' views and experiences of formal group partnerships remain unclear. In this qualitative evidence synthesis (QES), we focus specifically on formal group partnerships where health providers and consumers share decision-making about planning, delivering and/or evaluating health services. Formal group partnerships were selected because they are widely used throughout the world to improve person-centred care. For the purposes of this QES, the term 'consumer' refers to a person who is a patient, carer or community member who brings their perspective to health service partnerships. 'Health provider' refers to a person with a health policy, management, administrative or clinical role who participates in formal partnerships in an advisory or representative capacity. This QES was co-produced with a Stakeholder Panel of consumers and health providers. The QES was undertaken concurrently with a Cochrane intervention review entitled Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation.

OBJECTIVES

1. To synthesise the views and experiences of consumers and health providers of formal partnership approaches that aimed to improve planning, delivery or evaluation of health services. 2. To identify best practice principles for formal partnership approaches in health services by understanding consumers' and health providers' views and experiences.

SEARCH METHODS

We searched MEDLINE, Embase, PsycINFO and CINAHL for studies published between January 2000 and October 2018. We also searched grey literature sources including websites of relevant research and policy organisations involved in promoting person-centred care.

SELECTION CRITERIA

We included qualitative studies that explored consumers' and health providers' perceptions and experiences of partnering in formal group formats to improve the planning, delivery or evaluation of health services.

DATA COLLECTION AND ANALYSIS

Following completion of abstract and full-text screening, we used purposive sampling to select a sample of eligible studies that covered a range of pre-defined criteria, including rich data, range of countries and country income level, settings, participants, and types of partnership activities. A Framework Synthesis approach was used to synthesise the findings of the sample. We appraised the quality of each study using the CASP (Critical Appraisal Skill Program) tool. We assessed our confidence in the findings using the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach. The Stakeholder Panel was involved in each stage of the review from development of the protocol to development of the best practice principles.

MAIN RESULTS

We found 182 studies that were eligible for inclusion. From this group, we selected 33 studies to include in the final synthesis. These studies came from a wide range of countries including 28 from high-income countries and five from low- or middle-income countries (LMICs). Each of the studies included the experiences and views of consumers and/or health providers of partnering in formal group formats. The results were divided into the following categories. Contextual factors influencing partnerships: government policy, policy implementation processes and funding, as well as the organisational context of the health service, could facilitate or impede partnering (moderate level of confidence). Consumer recruitment: consumer recruitment occurred in different ways and consumers managed the recruitment process in a minority of studies only (high level of confidence). Recruiting a range of consumers who were reflective of the clinic's demographic population was considered desirable, particularly by health providers (high level of confidence). Some health providers perceived that individual consumers' experiences were not generalisable to the broader population whereas consumers perceived it could be problematic to aim to represent a broad range of community views (high level of confidence). Partnership dynamics and processes: positive interpersonal dynamics between health providers and consumers facilitated partnerships (high level of confidence). However, formal meeting formats and lack of clarity about the consumer role could constrain consumers' involvement (high level of confidence). Health providers' professional status, technical knowledge and use of jargon were intimidating for some consumers (high level of confidence) and consumers could feel their experiential knowledge was not valued (moderate level of confidence). Consumers could also become frustrated when health providers dominated the meeting agenda (moderate level of confidence) and when they experienced token involvement, such as a lack of decision-making power (high level of confidence) Perceived impacts on partnership participants: partnering could affect health provider and consumer participants in both positive and negative ways (high level of confidence). Perceived impacts on health service planning, delivery and evaluation: partnering was perceived to improve the person-centredness of health service culture (high level of confidence), improve the built environment of the health service (high level of confidence), improve health service design and delivery e.g. facilitate 'out of hours' services or treatment closer to home (high level of confidence), enhance community ownership of health services, particularly in LMICs (moderate level of confidence), and improve consumer involvement in strategic decision-making, under certain conditions (moderate level of confidence). There was limited evidence suggesting partnering may improve health service evaluation (very low level of confidence). Best practice principles for formal partnering to promote person-centred care were developed from these findings. The principles were developed collaboratively with the Stakeholder Panel and included leadership and health service culture; diversity; equity; mutual respect; shared vision and regular communication; shared agendas and decision-making; influence and sustainability.

AUTHORS' CONCLUSIONS

Successful formal group partnerships with consumers require health providers to continually reflect and address power imbalances that may constrain consumers' participation. Such imbalances may be particularly acute in recruitment procedures, meeting structure and content and decision-making processes. Formal group partnerships were perceived to improve the physical environment of health services, the person-centredness of health service culture and health service design and delivery. Implementing the best practice principles may help to address power imbalances, strengthen formal partnering, improve the experiences of consumers and health providers and positively affect partnership outcomes.

Health promotion in secondary schools: participatory process for constructing a self-assessment tool

This article presents the process of constructing an effective self-assessment tool for monitoring health promotion in secondary schools (SS). The ultimate aim is to improve adolescents' well-being and quality of life in their reference environment. The design and validation of the instrument were based on participatory action research (involving students, professors, parents, health professionals, youth workers, youth directors, youth managers). A mixed quantitative and qualitative approach was adopted. The construction process included the following five consecutive stages: (i) steering-group creation and definition of objectives; (ii) diagnosing students' health and well-being in SS (case study); (iii) initial design and construction of health promotion indicators; (iv) validation based on expert judgement and (v) validation by means of a pilot test. The final construction of the self-assessment tool included 5 areas (healthy habits, affectivity and socialization, emotional well-being, safety and risks, and specific health situations), 9 objectives and 18 indicators deployed through a rubric. Each indicator is discussed in depth, specifying the most appropriate resources and health promotion activities for its application. The final self-assessment tool is designed to be sensitive to and effective for self-assessment of health promotion in SS. It contributes to improving adolescent health on an individual basis and also has an impact on the school environment. Finally, it also promotes analysis of the health content on the curriculum and the teaching-learning method(s) employed at the school. The self-assessment tool has been published open access and its implementation will continue to increase health promotion in SS in Catalonia.

Meta-evaluation of a participatory process in the strengthening of municipal management

OBJECTIVE

To evaluate, with a focus on participation, an evaluation process developed by municipal managers and administrators of a health region in the state of São Paulo, considering the need for theoretical reflection on participatory health practices in the Brazilian context.

METHODS

Qualitative research that used the framework developed by Daigneault and Jacob (2009) to analyze the empirical material, encompassing three dimensions of participation: control of the evaluation process, diversity of participants, and extent of their involvement. We highlighted decisions or contextual aspects that deepened or limited the participatory option in the process under study.

RESULTS

We identified the presence and important performance of stakeholders who are “not specialists in evaluation”, through participation both in the direction of the evaluation process and in its distinct stages. The formed group started from their own annoyances added to the need for information and reflection to define the subject and scope of the evaluation; the use of the process planned by them guided the definition of the data to be collected and the format of result dissemination; the empirical material analysis was undertaken jointly by the participants. Regarding the third dimension, a limitation was identified regarding the diversity of actors involved due to the prioritization of the possibility of in-depth work with a fixed group of managers.

CONCLUSIONS

It is stated that there is no “ideal participation model” for evaluations. In certain contexts and structures, real opportunities for participation – even if they seem fragile at first sight – should be leveraged, and that requires flexibility and critical reflection on the part of those responsible for the evaluation processes to undertake the necessary adjustments.