Using Best-Worst Scaling to Measure Caregiver Preferences for Managing their Child's ADHD: A Pilot Study

Preferences in the Design and Delivery of Neurodevelopmental Follow-Up Care for Children: A Systematic Review of Discrete Choice Experiments

Neurodevelopmental disorders are a significant cause of morbidity. Early detection of neurodevelopmental delay is essential for timely diagnosis and intervention, and it is therefore important to understand the preferences of parents and clinicians for engaging with neurodevelopmental surveillance and follow-up care. Discrete choice experiment (DCE) may be an appropriate method for quantifying these preferences. This review systematically examined how DCEs have been designed and delivered in studies examining neurodevelopmental care of children and identified the preferred attributes that have been reported. PubMed, Embase, CINAHL, and Scopus databases were systematically searched. Studies were included if they used DCE to elicit preferences for a neurodevelopmental follow-up program for children. Two independent reviewers conducted the title and abstract and full-text screening. Risk of bias was assessed using a DCE-specific checklist. Findings were presented using a narrative synthesis. A total of 6618 records were identified and 16 papers were included. Orthogonal (n=5) and efficient (n=5) experimental designs were common. There was inconsistent reporting of design-related features. Analysis was primarily completed using mixed logit (n=6) or multinomial logit (n=3) models. Several key attributes for neurodevelopmental follow-up care were identified including social, behavioral and emotional support, therapy, waiting time, and out-of-pocket costs. DCE has been successfully used as a preference elicitation method for neurodevelopmental-related care. There is scope for improvement in the design and analysis of DCE in this field. Nonetheless, attributes identified in these studies are likely to be important considerations in the design and implementation of programs for neurodevelopmental care.

Adult Patient Preferences for Long-Acting ADHD Treatments: A Discrete Choice Experiment

BACKGROUND AND OBJECTIVE

Treatment for attention deficit hyperactivity disorder (ADHD) requires a multifaceted approach including psychosocial interventions and pharmacological treatment. This study evaluates preferences for specific attributes associated with different long-acting stimulant treatment among US adults with ADHD.

METHODS

Patients completed an online, cross-sectional survey, incorporating a discrete choice experiment to assess preferences for attributes.

RESULTS

Analyses included 200 adults with ADHD (mean age 33.0 years; 60% self-reporting moderate severity); the mean (SD) Adult ADHD Self-Report Scale-v1.1 score was 45.9 (12.4). Overall, patients valued speed of onset most and risk of rebound least. Three population groups with distinct preferences were identified: side effect-driven (n=69, 35%), quick onset-driven (n=47, 24%) and quick onset and long duration-driven (n=84, 42%).

CONCLUSION

This study shows differences in how adults with ADHD value and assess benefit-risk trade-offs when considering the desired attributes of stimulant treatments, highlighting the importance of patient-physician shared decision-making to optimize the desired benefits of individualized treatment.

Health versus other sectors: Multisectoral resource allocation preferences in Mukono district, Uganda

Objectives

To elicit citizen preferences for national budget resource allocation in Uganda, examine respondents’ preferences for health vis-à-vis other sectors, and compare these preferences with actual government budget allocations.

Methods

We surveyed 432 households in urban and rural areas of Mukono district in central Uganda.We elicited citizens’ preferences for resource allocation across all sectors using a best-worst scaling (BWS) survey. The BWS survey consisted of 16 sectors corresponding to the Uganda national budget line items. Respondents chose, from a subset of four sectors across 16 choice tasks, which sectors they thought were most and least important to allocate resources to. We utilized the relative best-minus-worst score method and a conditional logistic regression to obtain ranked preferences for resource allocation across sectors. We then compared the respondents’ preferences with actual government budget allocations.

Results

The health sector was the top ranked sector where 82% of respondents selected health as the most important sector for the government to fund, but it was ranked sixth in national budget allocation, encompassing 6.4% of the total budget. Beyond health, water and environment, agriculture, and social development sectors were largely underfunded compared to respondents’ preferences. Works and transport, education, security, and justice, law and order received a larger share of the national budget compared to respondents’ preferences.

Conclusions

Among respondents from Mukono district in Uganda, we found that citizens’ preferences for resource allocation across sectors, including for the health sector, were fundamentally misaligned with current government budget allocations. Evidence of respondents’ strong preferences for allocating resources to the health sector could help stakeholders make the case for increased health sector allocations. Greater investment in health is not only essential to satisfy citizens’ needs and preferences, but also to meet the government’s health goals to improve health, strengthen health systems, and achieve universal health coverage.

Stakeholder-Engaged Derivation of Patient-Informed Value Elements

Objectives

Our objective was to identify patient-informed value elements that can be used to make value assessment more patient centered.

Methods

Mixed methods were used iteratively to collect and integrate qualitative and quantitative data in a four-stage process: identification (stage 1), prioritization (stage 2), refinement (stage 3), and synthesis (stage 4). Qualitative methods involved one-on-one discussions with 14 patient stakeholders from diverse medical communities representing mental health, osteoporosis, blindness, lupus, eczema, oncology, chronic obstructive pulmonary disease, and hypercholesterolemia. Stakeholders completed guided activities to prioritize elements important to patient healthcare decision making. Responses were summarized descriptively as frequencies and proportions.

Results

Stakeholders identified 94 value elements in stage 1. Of these, 42 elements remained following the stage 2 prioritization and the stage 3 refinement. During the stage 4 synthesis, the 42 patient-informed value elements comprised the principal set of value elements that were organized by 11 categories: tolerability, disease burden, forecasting, accessibility of care/treatment, healthcare service delivery, cost incurred on the patient, cost incurred on the family, personal well-being, stigma, social well-being, and personal values. The categories fell under five domains: short- and long-term effects of treatment, treatment access, cost, life impact, and social impact.

Conclusions

In total, 75% of the value elements in the conceptual model were patient derived and distinct from the elements used in existing value frameworks. Recommendations for tailoring, quantifying, and applying the patient-informed value elements in distinct patient communities are provided. This provides a foundation from which future research may test patient-informed value elements in existing value frameworks and economic evaluations.

Electronic supplementary material

The online version of this article (10.1007/s40271-020-00433-8) contains supplementary material, which is available to authorized users.

Care Management for Youth With Comorbid Developmental and Mental Health Conditions: A Discrete Choice Experiment Pilot Study

OBJECTIVE

Caregivers of a child with a coexisting cognitive/intellectual and an emotional/behavior/developmental disability have difficult decisions regarding care management options for their child. This study aimed to pilot and refine an instrument to elicit caregivers' preferences in managing their child's care needs.

METHODS

Subjects were 38 caregivers of a child aged 21 and younger with a coexisting cognitive/intellectual and an emotional/behavior/developmental disability. A mixed-methods design was used to develop and pilot a discrete choice experiment (DCE) to elicit care management preferences for their child. Six attributes of care management decisions were tested in the DCE: medication use, parental custody, time cost, social interactions, medication effects, and school placement. Subjects completed a paper-and-pencil survey after which a debriefing discussion was held to obtain feedback that would aid in refining the attribute descriptions. Conditional logistic regression generated mean scores for each attribute. Comments from the debriefing sessions were audio-recorded and used to modify the attribute descriptions.

RESULTS

The majority (84%) of subjects were aged 40 years or older and a female caregiver. Common diagnoses of the children were autism spectrum disorder (55%) and attention-deficit/hyperactivity disorder (76%). Subjects preferred using fewer medications and maintaining decision-making authority as opposed to delegating authority to a third party. Medication effects on the child's mood were more important than effects on personality or body weight.

CONCLUSIONS

The DCE was sensitive to caregivers' preferences for managing their child's coexisting cognitive/intellectual and emotional/behavior/developmental disability. Findings may help providers gauge treatment in a broader context of health outcomes.

Developing and piloting an instrument to prioritize the worries of patients with acute myeloid leukemia

BACKGROUND

Acute myeloid leukemia (AML) is a rapidly progressing blood cancer for which new treatments are needed. We sought to promote patient-focused drug development (PFDD) for AML by developing and piloting an instrument to prioritize the worries of patients with AML.

PATIENTS AND METHODS

An innovative community-centered approach was used to engage expert and community stakeholders in the development, pretesting, pilot testing, and dissemination of a novel best-worst scaling instrument. Patient worries were identified through individual interviews (n=15) and group calls. The instrument was developed through rigorous pretesting (n=13) and then piloted among patients and caregivers engaged in this study (n=25). Priorities were assessed using best-worst scores (spanning from +1 to -1) representing the relative number of times that items were endorsed as the most and the least worrying. All findings were presented at a PFDD meeting at the US Food and Drug Administration (FDA) that was attended by >80 stakeholders.

RESULTS

The final instrument included 13 worries spanning issues such as decision making, treatment delivery, physical impacts, and psychosocial effects. Patients and caregivers most prioritized worries about dying from their disease (best minus worst [BW] score=0.73), long-term side effects (BW=0.28), and time in hospital (BW=0.25).

CONCLUSION

Community-centered approaches are valuable in designing and executing PFDD meetings and associated quantitative surveys to document the experience of patients. Expert and community stakeholders welcomed the opportunity to share their experiences with the FDA and strongly endorsed implementing this survey nationally.

Development of a stated-preference instrument to prioritize treatment goals in recent onset schizophrenia

OBJECTIVE

This study sought to evaluate a new stated-preference instrument to prioritize multiple treatment goals among people with recent onset schizophrenia.

METHODS

A draft survey instrument was developed to assess preferences for 13 key treatment goals that were identified based on the literature. The survey incorporates best-worst scaling (BWS), which shows repeated subsets comprising 4 of the 13 goals, and respondents identify which is most important and which is least important to them. Pre-test interviews were conducted in the UK, Italy, and Germany among people aged 18 to 35 diagnosed with schizophrenia within the past 5 years. Specifically, participants completed the instrument online in their native language, followed by a telephone interview to provide feedback on the clarity, relevance, and comprehensiveness of the survey. The interview data were analyzed to assess interpretation and content validity of the survey.

RESULTS

Fifteen participants (five per country) provided feedback (mean age = 31; 60% male). Feedback was comparable across countries and confirmed that the key treatment goals assessed were relevant and meaningful. Probing by interviewers ascertained that respondents interpreted the questions appropriately and identified the treatment goals that were most and least important to them. Based on the characterization of the goals, a conceptual model was developed illustrating hypothesized associations among them.

CONCLUSION

The results confirm that the BWS methodology and key treatment goals in this new instrument are appropriate for use in recent onset schizophrenia. These results will help guide measurement of patient-relevant endpoints in future studies.

A Latent Class Analysis to Identify Variation in Caregivers' Preferences for their Child's Attention-Deficit/Hyperactivity Disorder Treatment: Do Stated Preferences Match Current Treatment?

OBJECTIVES

To investigate variation in caregiver preferences for their child's attention-deficit/hyperactivity disorder (ADHD) care and to determine if their stated preferences align with current care management.

METHODS

Caregivers of a child aged 4-14 years and in care for ADHD were recruited from pediatric outpatient clinics and advocacy groups across the state of Maryland. Participants completed a survey collecting demographics, the child's treatment, and caregiver preferences-elicited using a best-worst scaling experiment (case 2). Latent class analysis was used to identify distinct preference segments and bivariate analyses were used to compare the association between segment membership with what the child was currently receiving for their ADHD.

RESULTS

Participants (n = 184) were predominantly White (68%) and the child's mother (84%). Most children had ADHD for 2 or more years (79%). Caregiver preferences were distinguished by two segments: continuous medication (36%) and minimal medication (64%). The two groups had very different preferences for when medication was administered (p < 0.001), but they had similar preferences for provider-oriented and non-medication interventions (p > 0.05 for the caregiver behavior training, provider communication, provider specialty, and out-of-pocket costs). One third of the sample did not receive the preferred individualized education program and 42% of the minimal medication group reported using medication 7 days a week all year round.

CONCLUSIONS

Although behavior management training and school accommodations aspects of an ADHD care plan are more important to caregivers than evidence-based medication, fewer families had access to educational accommodations. Further research is needed to clarify how stated preferences for care align with treatments used in actual practice settings.

Caregiver Treatment Preferences for Children with a New Versus Existing Attention-Deficit/Hyperactivity Disorder Diagnosis

OBJECTIVES

Parental experiences with managing their child's attention-deficit/hyperactivity disorder (ADHD) can influence priorities for treatment. This study aimed to identify the ADHD management options caregivers most prefer and to determine if preferences differ by time since initial ADHD diagnosis.

METHODS

Primary caregivers (n = 184) of a child aged 4-14 years old in care for ADHD were recruited from January 2013 through March 2015 from community-based pediatric and mental health clinics and family support organizations across the state of Maryland. Participants completed a survey that included child/family demographics, child clinical treatment, and a Best-Worst Scaling (BWS) experiment to elicit ADHD management preferences. The BWS comprised 18 ADHD management profiles showing seven treatment attributes, where the best and worst attribute levels were selected from each profile. A conditional logit model using effect-coded variables was used to estimate preference weights stratified by time since ADHD diagnosis.

RESULTS

Participants were primarily the mother (84%) and had a college or postgraduate education (76%) with 75% of the children on stimulant medications. One-on-one caregiver behavior training, medication use seven days a week, therapy in a clinic, and an individualized education program were most preferred for managing ADHD. Aside from caregiver training and monthly out-of-pocket costs, caregivers of children diagnosed with ADHD for less than two years prioritized medication use lower than other care management attributes and caregivers of children diagnosed with ADHD for two or more years preferred school accommodations, medication, and provider specialty.

CONCLUSIONS

Preferences for ADHD treatment differ based on the duration of the child's ADHD. Acknowledging that preferences change over the course of care could facilitate patient/family-centered care planning across a range of resources and a multidisciplinary team of professionals.

Experimental measurement of preferences in health and healthcare using best-worst scaling: an overview

Best-worst scaling (BWS), also known as maximum-difference scaling, is a multiattribute approach to measuring preferences. BWS aims at the analysis of preferences regarding a set of attributes, their levels or alternatives. It is a stated-preference method based on the assumption that respondents are capable of making judgments regarding the best and the worst (or the most and least important, respectively) out of three or more elements of a choice-set. As is true of discrete choice experiments (DCE) generally, BWS avoids the known weaknesses of rating and ranking scales while holding the promise of generating additional information by making respondents choose twice, namely the best as well as the worst criteria. A systematic literature review found 53 BWS applications in health and healthcare. This article expounds possibilities of application, the underlying theoretical concepts and the implementation of BWS in its three variants: 'object case', 'profile case', 'multiprofile case'. This paper contains a survey of BWS methods and revolves around study design, experimental design, and data analysis. Moreover the article discusses the strengths and weaknesses of the three types of BWS distinguished and offered an outlook. A companion paper focuses on special issues of theory and statistical inference confronting BWS in preference measurement.

Using Best-Worst Scaling to Investigate Preferences in Health Care

INTRODUCTION

Best-worst scaling (BWS) is becoming increasingly popular to elicit preferences in health care. However, little is known about current practice and trends in the use of BWS in health care. This study aimed to identify, review and critically appraise BWS in health care, and to identify trends over time in key aspects of BWS.

METHODS

A systematic review was conducted, using Medline (via Pubmed) and EMBASE to identify all English-language BWS studies published up until April 2016. Using a predefined extraction form, two reviewers independently selected articles and critically appraised the study quality, using the Purpose, Respondents, Explanation, Findings, Significance (PREFS) checklist. Trends over time periods (≤2010, 2011, 2012, 2013, 2014 and 2015) were assessed further.

RESULTS

A total of 62 BWS studies were identified, of which 26 were BWS object case studies, 29 were BWS profile case studies and seven were BWS multi-profile case studies. About two thirds of the studies were performed in the last 2 years. Decreasing sample sizes and decreasing numbers of factors in BWS object case studies, as well as use of less complicated analytical methods, were observed in recent studies. The quality of the BWS studies was generally acceptable according to the PREFS checklist, except that most studies did not indicate whether the responders were similar to the non-responders.

CONCLUSION

Use of BWS object case and BWS profile case has drastically increased in health care, especially in the last 2 years. In contrast with previous discrete-choice experiment reviews, there is increasing use of less sophisticated analytical methods.

Systematic Review of Patients' and Parents' Preferences for ADHD Treatment Options and Processes of Care

BACKGROUND

Patient preferences are an important topic of study with respect to attention-deficit hyperactivity disorder (ADHD) interventions, as there are multiple treatment choices available, multiple developmental levels to consider, and multiple potential individuals involved in treatment (children, parents, and adults with ADHD). Stated preference methods such as discrete choice experiment (DCE), best-worst scaling (BWS), and other utility value methods such as standard gamble interview (SGI) and time trade-off (TTO) are becoming more common in research addressing preferences for ADHD treatments. A synthesis of this research may facilitate improved patient-centered and family-centered treatment for ADHD.

OBJECTIVE

The purpose of this review was to synthesize reports across existing DCE, BWS, TTO, and SGI studies to assess which aspects of ADHD treatment are most studied as well as most preferred and influential in treatment decisions.

DATA SOURCES

MEDLINE, PsycINFO.

STUDY SELECTION

A total of 41 studies referring to preferences for ADHD treatment were identified through the initial search and contact with researchers. Of these, 13 reported ADHD treatment preference data from a study using DCE, BWS, or SGI methods. No TTO studies were identified that met inclusion criteria.

RESULTS

Methods and designs varied considerably across studies. Relatively few studies focused on preferences among children, adolescents, and adults compared with those that focused on the preferences of parents of children with ADHD. The majority of studies focused primarily on medication treatments, with many fewer focused on psychosocial treatments. Some studies indicated that parents of children with ADHD prefer to avoid stimulant medications in favor of behavioral or psychosocial interventions. Others report that parents see medication as a preferred treatment. Treatment outcome is a particularly salient attribute for treatment decisions for many informants.

CONCLUSIONS

Potential outcomes of various treatments play a proximal role in patients' and families' decisions for ADHD treatment. Because the majority of studies focus on medication treatments for children with ADHD, more research is necessary to understand preferences related to behavioral and other psychosocial treatments both as stand-alone interventions and used in combination with medication. Additional research is also needed to assess the treatment preferences of adults with ADHD. In general, DCE, BWS, and SGI methods allow measurement of patient preferences in a manner that approximates the uncertainty and trade-offs inherent in real-world treatment decision making and provides valuable information to inform patient-centered and family-centered treatment.

Using Best-Worst Scaling to Measure Caregiver Preferences for Managing their Child's ADHD: A Pilot Study

BACKGROUND

Management of attention-deficit/hyperactivity disorder (ADHD) is a trade-off between caregivers' concerns about the benefits versus the risks of evidence-based treatment. Few studies have used choice-based methods to assess what treatment attributes matter most to caregivers.

OBJECTIVE

The aim was to develop and to pilot an instrument to elicit caregivers' preferences for evidence-based management of their child's ADHD.

METHODS

Mixed methods were used to develop a Best-Worst Scaling (BWS) instrument, and quantitative methods were used to pilot the instrument. Primary caregivers of children with ADHD from two community organizations were recruited for the development (n = 21) and pilot (n = 37) phase. The instrument was a BWS case 2, where 18 management profiles are presented one at a time, with respondents indicating the one best and one worst feature of each profile. Profiles were developed using a main effects orthogonal array. The mean of best-minus-worse scores was estimated, and attribute importance was based on the sum of maximum minus minimum scores for each attribute. Feasibility of eliciting stated preferences was evaluated with t tests and 95 % confidence intervals.

RESULTS

Seven attributes (medication, therapy, school, caregiver training, provider specialty, provider communication, and out-of-pocket costs) with three levels each were identified. All mean scores were significant except for pediatrician management of the child's ADHD (p = 0.089). Caregiver training had the highest relative importance, followed by medication and provider communication.

CONCLUSIONS

The BWS instrument was a relatively simple measure, caregivers completed it independently, and it distinguished the relative importance of different attributes in managing a child's ADHD.