Modeling the Decision of Mental Health Providers to Implement Evidence-Based Children's Mental Health Services: A Discrete Choice Conjoint Experiment

Current Practices for Accounting for Preference Heterogeneity in Health-Related Discrete Choice Experiments: A Systematic Review

BACKGROUND

Accounting for preference heterogeneity is a growing analytical practice in health-related discrete choice experiments (DCEs). As heterogeneity may be examined from different stakeholder perspectives with different methods, identifying the breadth of these methodological approaches and understanding the differences are major steps to provide guidance on good research practices.

OBJECTIVES

Our objective was to systematically summarize current practices that account for preference heterogeneity based on the published DCEs related to healthcare.

METHODS

This systematic review is part of the project led by the Professional Society for Health Economics and Outcomes Research (ISPOR) health preference research special interest group. The systematic review conducted systematic searches on the PubMed, OVID, and Web of Science databases, as well as on two recently published reviews, to identify articles. The review included health-related DCE articles published between 1 January 2000 and 30 March 2020. All the included articles also presented evidence on preference heterogeneity analysis based on either explained or unexplained factors or both.

RESULTS

Overall, 342 of the 2202 (16%) articles met the inclusion/exclusion criteria for extraction. The trend showed that analyses of preference heterogeneity increased substantially after 2010 and that such analyses mainly examined heterogeneity due to observable or unobservable factors in individual characteristics. Heterogeneity through observable differences (i.e., explained heterogeneity) is identified among 131 (40%) of the 342 articles and included one or more interactions between an attribute variable and an observable characteristic of the respondent. To capture unobserved heterogeneity (i.e., unexplained heterogeneity), the studies largely estimated either a mixed logit (n = 205, 60%) or a latent-class logit (n = 112, 32.7%) model. Few studies (n = 38, 11%) explored scale heterogeneity or heteroskedasticity.

CONCLUSIONS

Providing preference heterogeneity evidence in health-related DCEs has been found as an increasingly used practice among researchers. In recent studies, controlling for unexplained preference heterogeneity has been seen as a common practice rather than explained ones (e.g., interactions), yet a lack of providing methodological details has been observed in many studies that might impact the quality of analysis. As heterogeneity can be assessed from different stakeholder perspectives with different methods, researchers should become more technically pronounced to increase confidence in the results and improve the ability of decision makers to act on the preference evidence.

Understanding the value of adhering to or adapting evidence-based interventions: a study protocol of a discrete choice experiment

Background

Whereas the value of an evidence-based intervention (EBI) is often determined by its effect on clinical outcomes, the value of implementing and using EBIs in practice is broader, reflecting qualities such as appropriateness, equity, costs, and impact. Reconciling these value conflicts involves a complicated decision process that has received very limited scholarly attention. Inspired by studies on decision-making, the objective of this project is to explore how practitioners appraise the values of different outcomes and to test how this appraisal influences their decisions surrounding the so-called fidelity–adaptation dilemma. This dilemma is related to the balance between using an EBI as it was designed (to ensure its effectiveness) and making appropriate adaptations (to ensure alignment with constraints and possibilities in the local context).

Methods

This project consists of three sub-studies. The participants will be professionals leading evidence-based parental programs in Sweden and, in Sub-study 1, parents and decision-makers. Sub-study 1 will use sequential focus groups and individual interviews to explore parameters that influence fidelity and adaptation decisions—the dilemmas encountered, available options, how outcomes are valued by practitioners as well as other stakeholders, and value trade-offs. Sub-study 2 is a discrete choice experiment that will test how value appraisals influence decision-making using data from Sub-study 1 as input. Sub-study 3 uses a mixed-method design, with findings from the two preceding sub-studies as input in focus group interviews to investigate how practitioners make sense of findings from optimal decision situations (experiment) and constrained, real-world decision situations.

Discussion

The project will offer unique insights into decision-making processes that influence how EBIs are used in practice. Such knowledge is needed for a more granular understanding of how practitioners manage the fidelity–adaptation dilemma and thus, ultimately, how the value of EBI implementation can be optimized. This study contributes to our knowledge of what happens once EBIs are adopted—that is, the gap between the way in which EBIs are intended to be used and the way in which they are used in practice.

Mental health service preferences of patients and providers: a scoping review of conjoint analysis and discrete choice experiments from global public health literature over the last 20 years (1999-2019)

BACKGROUND

In designing, adapting, and integrating mental health interventions, it is pertinent to understand patients' needs and their own perceptions and values in receiving care. Conjoint analysis (CA) and discrete choice experiments (DCEs) are survey-based preference-elicitation approaches that, when applied to healthcare settings, offer opportunities to quantify and rank the healthcare-related choices of patients, providers, and other stakeholders. However, a knowledge gap exists in characterizing the extent to which DCEs/CA have been used in designing mental health services for patients and providers.

METHODS

We performed a scoping review from the past 20 years (2009-2019) to identify and describe applications of conjoint analysis and discrete choice experiments. We searched the following electronic databases: Pubmed, CINAHL, PsychInfo, Embase, Cochrane, and Web of Science to identify stakehold,er preferences for mental health services using Mesh terms. Studies were categorized according to pertaining to patients, providers and parents or caregivers.

RESULTS

Among the 30 studies we reviewed, most were published after 2010 (24/30, 80%), the majority were conducted in the United States (11/30, 37%) or Canada (10/30, 33%), and all were conducted in high-income settings. Studies more frequently elicited preferences from patients or potential patients (21/30, 70%) as opposed to providers. About half of the studies used CA while the others utilized DCEs. Nearly half of the studies sought preferences for mental health services in general (14/30, 47%) while a quarter specifically evaluated preferences for unipolar depression services (8/30, 27%). Most of the studies sought stakeholder preferences for attributes of mental health care and treatment services (17/30, 57%).

CONCLUSIONS

Overall, preference elicitation approaches have been increasingly applied to mental health services globally in the past 20 years. To date, these methods have been exclusively applied to populations within the field of mental health in high-income countries. Prioritizing patients' needs and preferences is a vital component of patient-centered care - one of the six domains of health care quality. Identifying patient preferences for mental health services may improve quality of care and, ultimately, increase acceptability and uptake of services among patients. Rigorous preference-elicitation approaches should be considered, especially in settings where mental health resources are scarce, to illuminate resource allocation toward preferred service characteristics especially within low-income settings.

Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment

BACKGROUND

Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling.

METHODS

A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles.

RESULTS

On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics.

CONCLUSIONS

The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

Stakeholder Coalitions and Priorities Around the Policy Goals of a Nation-Wide Mental Health Care Reform

The difficulty of implementing mental healthcare reforms owes much to the influence of stakeholders. So far, the endorsement of mental health policy reforms by stakeholder coalitions has received little attention. This study describes stakeholder coalitions formed around common mental health policy goals and highlights their central goals and oppositions. Data were collected on the policy priorities of 469 stakeholders (policymakers, service managers, clinicians, and user representatives) involved in the Belgian mental healthcare reform. Four coalitions of stakeholders endorsing different mental health policy goals were identified using a hierarchical cluster analysis on stakeholders' policy priorities. A belief network analysis was performed to identify the central and peripheral policy goals within coalitions. Coalitions brought together stakeholders with similar professional functions. Disagreements were observed between service managers and policymakers around policy goals. The two coalitions composed of policymakers supported a comprehensive approach that combines the different goals and also supported the shortening of hospital stays, whereas the two coalitions composed of service managers emphasised the personal recovery of users and continuity of care. Regardless of the coalitions' differing policy priorities, strengthening community care was a central goal while patient-centred goals were peripheral. The competing policy positions of the coalitions identified may explain the slow and inconsistent pace of the Belgian mental healthcare reform. Strengthening community care may be an essential part of reaching consensus across coalitions. Finally, special care must be taken to ensure that patient-centred policy goals, such as social integration, are not set aside in favour of other goals.

Elucidating the influence of supervisors' roles on implementation climate

Background

Supervisors play an essential role in implementation by diffusing and synthesizing information, selling implementation, and translating top management’s project plans to frontline workers. Theory and emerging evidence suggest that through these roles, supervisors shape implementation climate—i.e., the degree to which innovations are expected, supported, and rewarded. However, it is unclear exactly how supervisors carry out each of these roles in ways that contribute to implementation climate—this represents a gap in the understanding of the causal mechanisms that link supervisors’ behavior with implementation climate. This study examined how supervisors’ performance of each of these roles influences three core implementation climate domains (expectations, supports, and rewards).

Materials and methods

A sequenced behavioral health screening, assessment, and referral intervention was implemented within a county-based child welfare agency. We conducted 6 focus groups with supervisors and frontline workers from implementing work units 6 months post-implementation (n = 51) and 1 year later (n = 40) (12 groups total). Participants were asked about implementation determinants, including supervision and implementation context. We audio-recorded, transcribed, and analyzed focus groups using an open coding process during which the importance of the supervisors’ roles emerged as a major theme. We further analyzed this code using concepts and definitions related to middle managers’ roles and implementation climate.

Results

In this work setting, supervisors (1) diffused information about the intervention proactively, and in response to workers’ questions, (2) synthesized information by tailoring it to workers’ individual needs, (3) translated top managements’ project plans into day-to-day tasks through close monitoring and reminders, and (4) justified implementation. All four of these roles appeared to shape the implementation climate by conveying strong expectations for implementation. Three roles (diffusing, synthesizing, and mediating) influenced climate by supporting workers during implementation. Only one role (diffusing) influenced climate by conveying rewards.

Conclusions

Supervisors shaped implementation climate by carrying out four roles (diffusing, synthesizing, mediating, and selling). Findings suggest that the interaction of these roles convey expectations and support for implementation (two implementation climate domains). Our study advances the causal theory explaining how supervisors’ behavior shapes the implementation climate, which can inform implementation practice.

Understanding Innovation in Homeless Service Provision: A Study of Frontline Providers' Values-Readiness for Change

Service innovation for adults experiencing mental illness and homelessness typically involves shifting from treatment-led, staircase models toward recovery-oriented, Housing-First models. Aligning frontline service providers' values to those embedded within newer models is an important, but under-investigated, influence on the innovation process. To assess values alignment in this context, we conducted semi-structured qualitative interviews with frontline providers in staircase services in Ireland (n = 50). Data showed that, while their values mostly aligned to the treatment-led model, there was meaningful evidence of more recovery-oriented values, too. Strategies to enhance innovation through values-alignment are discussed.