1
|
Hirt J, Janiaud P, Düblin P, Hemkens LG. Meta-research on pragmatism of randomized trials: rationale and design of the PragMeta database. Trials 2023; 24:437. [PMID: 37391755 DOI: 10.1186/s13063-023-07474-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 06/24/2023] [Indexed: 07/02/2023] Open
Abstract
BACKGROUND Pragmatic trials provide decision-oriented, real-world evidence that is highly applicable and generalizable. The interest in real-world evidence is fueled by the assumption that effects in the "real-world" are different to effects obtained under artificial, controlled, research conditions as often used for traditional explanatory trials. However, it is unknown which features of pragmatism, generalizability, and applicability would be responsible for such differences. There is a need to provide empirical evidence and promote meta-research to answer these fundamental questions on the pragmatism of randomized trials and real-world evidence. Here, we describe the rationale and design of the PragMeta database which pursues this goal ( www.PragMeta.org ). METHODS PragMeta is a non-commercial, open data platform and infrastructure to facilitate research on pragmatic trials. It collects and shares data from published randomized trials that either have a specific design feature or other characteristic related to pragmatism or they form clusters of trials addressing the same research question but having different aspects of pragmatism. This lays the foundation to determine the relationship of various features of pragmatism, generalizability, and applicability with intervention effects or other trial characteristics. The database contains trial data actively collected for PragMeta but also allows to import and link existing datasets of trials collected for other purposes, forming a large-scale meta-database. PragMeta captures data on (1) trial and design characteristics (e.g., sample size, population, intervention/comparison, outcome, longitudinal structure, blinding), (2) effects estimates, and (3) various determinants of pragmatism (e.g., the use of routinely collected data) and ratings from established tools used to determine pragmatism (e.g., the PRagmatic-Explanatory Continuum Indicator Summary 2; PRECIS-2). PragMeta is continuously provided online, inviting the meta-research community to collaborate, contribute, and/or use the database. As of April 2023, PragMeta contains data from > 700 trials, mostly with assessments on pragmatism. CONCLUSIONS PragMeta will inform a better understanding of pragmatism and the generation and interpretation of real-world evidence.
Collapse
Affiliation(s)
- Julian Hirt
- Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Spitalstrasse 2, Basel, CH-4031, Switzerland
- Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
- Department of Health, Institute of Nursing Science, Eastern Switzerland University of Applied Sciences, St.Gallen, Switzerland
| | - Perrine Janiaud
- Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Spitalstrasse 2, Basel, CH-4031, Switzerland
- Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
| | - Pascal Düblin
- Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Spitalstrasse 2, Basel, CH-4031, Switzerland
| | - Lars G Hemkens
- Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Spitalstrasse 2, Basel, CH-4031, Switzerland.
- Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland.
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.
- Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany.
| |
Collapse
|
2
|
Palin V, Van Staa TP, Steels S, Troxel AB, Groenwold RHH, MacDonald TM, Torgerson D, Faries D, Mancini P, Ouwens M, Frith LJ, Tsirtsonis K, MacLennan G, Nordon C. A first step towards best practice recommendations for the design and statistical analyses of pragmatic clinical trials: a modified Delphi approach. Br J Clin Pharmacol 2022; 88:5183-5201. [PMID: 35701368 DOI: 10.1111/bcp.15441] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Revised: 04/29/2022] [Accepted: 05/22/2022] [Indexed: 11/30/2022] Open
Abstract
AIM Pragmatic clinical trials (PCTs) are randomised trials implemented through routine clinical practice, where design parameters of traditional randomised controlled trials are modified to increase generalizability. However, this may introduce statistical challenges. We aimed to identify these challenges and discuss possible solutions leading to best practice recommendations for the design and analysis of PCTs. METHODS A modified Delphi method was used to reach consensus among a panel of 11 experts in clinical trials and statistics. Statistical issues were identified in a focused literature review and aggregated with insights and possible solutions from expert collected through a series of survey iterations. Issues were ranked according to their importance. RESULTS 27 articles were included and combined with experts' insight to generate a list of issues categorized into: participants; recruiting sites; randomisation, blinding and intervention; outcome (selection and measurement); and data analysis. Consensus was reached about the most important issues: risk of participants' attrition; heterogeneity of "usual care" across sites; absence of blinding; use of a subjective endpoint; and data analysis aligned with the trial estimand. Potential issues should be anticipated and preferably be addressed in the trial protocol. The experts provided solutions regarding data collection and data analysis, which were considered of equal importance. DISCUSSION A set of important statistical issues in PCTs was identified and approaches were suggested to anticipate and/or minimize these through data analysis. Any impact of choosing a pragmatic design feature should be gauged in the light of the trial estimand.
Collapse
Affiliation(s)
- Victoria Palin
- Division of Informatics, Imaging & Data Sciences, Manchester Environmental Research Institute, University of Manchester, United Kingdom
| | - Tjeerd P Van Staa
- Division of Informatics, Imaging & Data Sciences, Manchester Environmental Research Institute, University of Manchester, United Kingdom
| | - Stephanie Steels
- Department of Social Care and Social Work, Manchester Metropolitan University, Manchester, United Kingdom
| | - Andrea B Troxel
- Division of Biostatistics, Department of Population Health, NYU Grossman School of Medicine, NYU, USA
| | - Rolf H H Groenwold
- Department of Clinical Epidemiology, Leiden University Medical Centre, The Netherlands
| | - Tom M MacDonald
- MEMO Research, University of Dundee, Ninewells Hospital & Medical School, Dundee, United Kingdom
| | - David Torgerson
- Department of Health Sciences, University of York, United Kingdom
| | - Douglas Faries
- Global Statistical Sciences, Eli Lilly & Co., Indianapolis, IN, USA
| | | | | | | | | | - Graham MacLennan
- The Centre for Healthcare Randomised Trials, University of Aberdeen, United Kingdom
| | - Clementine Nordon
- formally LASER Research, Paris, France; currently AstraZeneca, Cambridge, United Kingdom
| | | |
Collapse
|
3
|
Sandy LC, Glorioso TJ, Weinfurt K, Sugarman J, Peterson PN, Glasgow RE, Ho PM. Leave me out: Patients' characteristics and reasons for opting out of a pragmatic clinical trial involving medication adherence. Medicine (Baltimore) 2021; 100:e28136. [PMID: 34941059 PMCID: PMC8702195 DOI: 10.1097/md.0000000000028136] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 11/16/2021] [Indexed: 01/05/2023] Open
Abstract
Opt-out procedures are sometimes used instead of standard consent practices to enable patients to exercise their autonomous preferences regarding research participation while reducing patient and researcher burden. However, little is known about the characteristics of patients who opt-out of research and their reasons for doing so. We gathered such information in a large pragmatic clinical trial (PCT) evaluating the effect of theory informed text messages on medication adherence.Eligible patients, identified through electronic health records, were sent information about the study and provided with an opportunity to opt-out. Those opting out were asked to complete a voluntary survey regarding their reasons for doing so. Demographic data were compared among patients opting-out vs those included in the study using chi-squared tests and a log binomial regression model.Of 9046 patients receiving study packets, 906 (10.0%) patients returned opt-out forms. Of those, 451 (49.8%) returned the opt-out survey. Patients who opted out were more likely to be older, white, and nonHispanic than those who were included in the PCT. Survey respondents expressed high levels of trust in their health care providers, research, and system. Nearly half (46.6%) reported concerns about time as a reason to opt-out.In this PCT, 10% of patients receiving packets opted out, with significant differences in age, race, gender, and ethnicity compared to those included. Future trials should further investigate representativeness and reasons patients choose to opt-out of participating in research.
Collapse
Affiliation(s)
- Lisa Caputo Sandy
- General Internal Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO
- University of Colorado University of Colorado Anschutz Medical Campus, 13199 E Montview Blvd, Suite 300 Aurora, CO
| | | | - Kevin Weinfurt
- Department of Population and Health Sciences, Duke University, Durham, NC
| | - Jeremy Sugarman
- Berman Institute of Bioethics, Johns Hopkins University, Baltimore, MD
| | - Pamela N. Peterson
- Department of Internal Medicine, Denver Health and Hospital Authority, Denver, CO
- School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO
| | - Russell E. Glasgow
- Department of Family Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO
- Adult and Child Consortium for Health Outcomes Research and Delivery Science, University of Colorado Anschutz Medical Campus, Aurora, CO
| | - P. Michael Ho
- VA Eastern Colorado Health Care System, Aurora, CO
- School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO
| |
Collapse
|
4
|
Lawrence K, Rodriguez DV, Feldthouse DM, Shelley D, Yu JL, Belli HM, Gonzalez J, Tasneem S, Fontaine J, Groom LL, Luu S, Wu Y, McTigue KM, Rockette-Wagner B, Mann DM. Effectiveness of an Integrated Engagement Support System to Facilitate Patient Use of Digital Diabetes Prevention Programs: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2021; 10:e26750. [PMID: 33560240 PMCID: PMC7902197 DOI: 10.2196/26750] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Accepted: 01/05/2021] [Indexed: 01/16/2023] Open
Abstract
Background Digital diabetes prevention programs (dDPPs) are effective behavior change tools to prevent disease progression in patients at risk for diabetes. At present, these programs are poorly integrated into existing health information technology infrastructure and clinical workflows, resulting in barriers to provider-level knowledge of, interaction with, and support of patients who use dDPPs. Tools that can facilitate patient-provider interaction around dDPPs may contribute to improved patient engagement and adherence to these programs and improved health outcomes. Objective This study aims to use a rigorous, user-centered design (UCD) methodology to develop a theory-driven system that supports patient engagement with dDPPs and their primary care providers with their care. Methods This study will be conducted in 3 phases. In phase 1, we will use systematic UCD, Agile software development, and qualitative research methods to identify key user (patients, providers, clinical staff, digital health technologists, and content experts) requirements, constraints, and prioritization of high-impact features to design, develop, and refine a viable intervention prototype for the engagement system. In phase 2, we will conduct a single-arm feasibility pilot of the engagement system among patients with prediabetes and their primary care providers. In phase 3, we will conduct a 2-arm randomized controlled trial using the engagement system. Primary outcomes will be weight, BMI, and A1c at 6 and 12 months. Secondary outcomes will be patient engagement (use and activity) in the dDPP. The mediator variables (self-efficacy, digital health literacy, and patient-provider relationship) will be measured. Results The project was initiated in 2018 and funded in September 2019. Enrollment and data collection for phase 1 began in September 2019 under an Institutional Review Board quality improvement waiver granted in July 2019. As of December 2020, 27 patients have been enrolled and first results are expected to be submitted for publication in early 2021. The study received Institutional Review Board approval for phases 2 and 3 in December 2020, and phase 2 enrollment is expected to begin in early 2021. Conclusions Our findings will provide guidance for the design and development of technology to integrate dDPP platforms into existing clinical workflows. This will facilitate patient engagement in digital behavior change interventions and provider engagement in patients’ use of dDPPs. Integrated clinical tools that can facilitate patient-provider interaction around dDPPs may contribute to improved patient adherence to these programs and improved health outcomes by addressing barriers faced by both patients and providers. Further evaluation with pilot testing and a clinical trial will assess the effectiveness and implementation of these tools. Trial Registration ClinicalTrials.gov NCT04049500; https://clinicaltrials.gov/ct2/show/NCT04049500 International Registered Report Identifier (IRRID) DERR1-10.2196/26750
Collapse
Affiliation(s)
- Katharine Lawrence
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Danissa V Rodriguez
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Dawn M Feldthouse
- Clinical Systems & Clinical Transformation, Medical Center Information Technology Clinical Informatics Department, NYU Langone Health, New York, NY, United States
| | - Donna Shelley
- Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Jonathan L Yu
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Hayley M Belli
- Division of Biostatistics, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Javier Gonzalez
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Sumaiya Tasneem
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Jerlisa Fontaine
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Lisa L Groom
- NYU Rory Meyers College of Nursing, New York, NY, United States
| | - Son Luu
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Yinxiang Wu
- Division of Biostatistics, Department of Population Health, NYU Langone Health, New York, NY, United States
| | - Kathleen M McTigue
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, United States
| | - Bonny Rockette-Wagner
- Department of Epidemiology, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, United States
| | - Devin M Mann
- Healthcare Innovation Bridging Research, Informatics, and Design Lab, Department of Population Health, NYU Langone Health, New York, NY, United States
| |
Collapse
|
5
|
Affret A, Luc A, Baumann C, Bergman P, Le Faou AL, Pasquereau A, Arwidson P, Alla F, Cambon L. Effectiveness of the e-Tabac Info Service application for smoking cessation: a pragmatic randomised controlled trial. BMJ Open 2020; 10:e039515. [PMID: 33109670 PMCID: PMC7592285 DOI: 10.1136/bmjopen-2020-039515] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
OBJECTIVE To compare the effectiveness of the mobile e-Tabac Info Service (e-TIS) application (app) for helping adult smokers quit smoking with current practices. DESIGN Pragmatic randomised controlled trial with a 1-year follow-up (2017-2018). SETTING France, population-wide level. PARTICIPANTS 2806 adult smokers who wished to quit smoking were recruited via the website of the French National Mandatory Health Insurance fund. Of them, 1400 were randomised to the e-TIS app arm and 1406 were randomised to the current practices arm (control). INTERVENTION The app involved personalised interactive contacts that included questionnaires, advice, activities and text messages. All contacts were individually tailored and based on each smoker's progress.In the control group, recommended practices for quitting smoking were described on a non-interactive website. PRIMARY AND SECONDARY OUTCOMES MEASURES The primary outcome was 7-day point prevalence abstinence (PPA) at 6 months. The secondary outcomes included continuous abstinence rates at 6 and 12 months, minimum 24-hour point abstinence at 3 months, minimum 30-day point abstinence at 12 months and number and duration of quit attempts. RESULTS There was no difference between the e-TIS and control arms for the primary outcome (12.6% vs 13.7% for 7-day PPA at 6 months, p=0.3949, intention-to-treat analysis). However, e-TIS participants with high levels of exposure to the app, which was defined by the completion of at least eight activities or questionnaires, showed higher rates of smoking cessation than the control participants (17.6% vs 12.9% for 7-day PPA at 6 months, p=0.0169, per-protocol analysis). CONCLUSION Use of the e-TIS app was not associated with a higher rate of smoking cessation. However, high level of exposure to the e-TIS app may have been more effective than current practices. TRIAL REGISTRATION NUMBER NCT02841683.
Collapse
Affiliation(s)
- Aurélie Affret
- Population Health Research Center, UMR 1219, CIC-EC 1401, Université Bordeaux, Bordeaux, Nouvelle Aquitaine, France
| | | | | | - Pierre Bergman
- Caisse nationale de l'assurance maladie, Paris, Île-de-France, France
| | | | | | | | - François Alla
- Population Health Research Center, UMR 1219, CIC-EC 1401, Université Bordeaux, Bordeaux, Nouvelle Aquitaine, France
| | - Linda Cambon
- Population Health Research Center, UMR 1219, CIC-EC 1401, Université Bordeaux, Bordeaux, Nouvelle Aquitaine, France
| |
Collapse
|
6
|
Mercer T, Nulu S, Vedanthan R. Innovative Implementation Strategies for Hypertension Control in Low- and Middle-Income Countries: a Narrative Review. Curr Hypertens Rep 2020; 22:39. [PMID: 32405820 DOI: 10.1007/s11906-020-01045-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
PURPOSE OF REVIEW This review summarizes the most recent and innovative implementation strategies for hypertension control in low- and middle-income countries (LMICs). RECENT FINDINGS Implementation strategies from Latin America, Africa, and Asia were organized across three levels: community, health system, and policy/population. Multicomponent interventions involving task-shifting strategies, with or without mobile health tools, had the most supporting evidence, with policy or population-level interventions having the least, focused only on salt reduction with mixed results. More research is needed to better understand how context affects intervention implementation. There is an emerging evidence base for implementation strategies for hypertension control and CVD risk reduction in LMICs at the community and health system levels, but further research is needed to determine the most effective policy and population-level strategies. How to best account for local context in adapting and implementing these evidence-based interventions in LMICs still remains largely unknown. Accelerating the translation of this implementation research into policy and practice is imperative to improve health and save lives globally.
Collapse
Affiliation(s)
- Tim Mercer
- Department of Population Health, Division of Global Health, The University of Texas at Austin Dell Medical School, 1601 Trinity St., Bldg. B, Austin, TX, 78712, USA.
| | - Shanti Nulu
- Department of Internal Medicine, Division of Cardiology, The University of Texas at Austin Dell Medical School, Austin, TX, USA
| | - Rajesh Vedanthan
- Department of Population Health, NYU Grossman School of Medicine, New York, NY, USA
| |
Collapse
|
7
|
Ma Q, Chung H, Shambhu S, Roe M, Cziraky M, Jones WS, Haynes K. Administrative claims data to support pragmatic clinical trial outcome ascertainment on cardiovascular health. Clin Trials 2019; 16:419-430. [PMID: 31081367 DOI: 10.1177/1740774519846853] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
BACKGROUND/AIMS Health plan administrative claims data present a cost-effective complement to traditional trial-specific ascertainment of clinical events typically conducted through patient report or a single health system electronic health record. We aim to demonstrate the value of health plan claims data in improving the capture of endpoints in longitudinal pragmatic clinical trials. METHODS This retrospective cohort study paralleled the design of the ADAPTABLE (Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-Term Effectiveness) trial designed to compare the effectiveness of two doses of aspirin. We applied the ADAPTABLE identification query in claims data from Anthem, an American health insurance company, and identified health plan members who met the ADAPTABLE trial criteria. Among the ADAPTABLE eligible members, we selected overlapping members with PCORnet Clinical Data Research Networks in the 2 years prior to the index date (1 April 2014). PCORnet Clinical Data Research Networks consist of network partners (or healthcare systems) that store their electronic health record data in the same format to support multi-institutional research. ADAPTABLE outcome events-cardiovascular hospitalizations including admissions for myocardial infarction, stroke, or cardiac procedures; hospitalizations for major bleeding; and in-hospital deaths-were evaluated for a 2-year follow-up period. Events were classified as within or outside PCORnet Clinical Data Research Networks using facility identifiers affiliated with each hospital stay. Patient characteristics were examined with descriptive statistics, and incidence rates were reported for available Clinical Data Research Networks and claims data. RESULTS Among 884,311 ADAPTABLE eligible health plan members, 11,101 patients overlapped with PCORnet Clinical Data Research Networks. Average age was 70 years, 71% were male, and average follow-up was 20.7 months. Patients had 1521 cardiovascular hospitalizations (571 (37.5%) occurred outside PCORnet Clinical Data Research Networks), 710 for major bleeding (296 (41.7%) outside PCORnet Clinical Data Research Networks), and 196 in-hospital deaths (67 (34.2%) outside PCORnet Clinical Data Research Networks). Incidence rates (events per1000 patient-months) differed between available network partners and claims data: cardiovascular hospitalizations, 4.1 (95% confidence interval: 3.9, 4.4) versus 6.6 (95% confidence interval: 6.3, 7.0), major bleeding, 1.8 (95% confidence interval: 1.6, 2.0) versus 3.1 (95% confidence interval: 2.9, 3.3), and in-hospital death, 0.56 (95% confidence interval: 0.47, 0.67) versus 0.85 (95% confidence interval: 0.74, 0.98), respectively. CONCLUSION This study demonstrated the value of supplementing longitudinal site-based clinical studies with administrative claims data. Our results suggest that claims data together with network partner electronic health record data constitute an effective vehicle to capture patient outcomes since >30% of patients have non-fatal and fatal events outside of enrolling sites.
Collapse
Affiliation(s)
- Qinli Ma
- 1 HealthCore, Inc., Wilmington, DE, USA
| | | | | | - Matthew Roe
- 2 Duke Heart Center, Duke Clinical Research Institute, Duke University Medical Center, Durham, NC, USA
| | | | - W Schuyler Jones
- 2 Duke Heart Center, Duke Clinical Research Institute, Duke University Medical Center, Durham, NC, USA
| | | |
Collapse
|
8
|
Ioannidis JP. Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright. Soc Sci Med 2018; 210:53-56. [DOI: 10.1016/j.socscimed.2018.04.029] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
9
|
Beidas RS, Becker-Haimes EM, Adams DR, Skriner L, Stewart RE, Wolk CB, Buttenheim AM, Williams NJ, Inacker P, Richey E, Marcus SC. Feasibility and acceptability of two incentive-based implementation strategies for mental health therapists implementing cognitive-behavioral therapy: a pilot study to inform a randomized controlled trial. Implement Sci 2017; 12:148. [PMID: 29246236 PMCID: PMC5732393 DOI: 10.1186/s13012-017-0684-7] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2017] [Accepted: 11/27/2017] [Indexed: 11/28/2022] Open
Abstract
BACKGROUND Informed by our prior work indicating that therapists do not feel recognized or rewarded for implementation of evidence-based practices, we tested the feasibility and acceptability of two incentive-based implementation strategies that seek to improve therapist adherence to cognitive-behavioral therapy for youth, an evidence-based practice. METHODS This study was conducted over 6 weeks in two community mental health agencies with therapists (n = 11) and leaders (n = 4). Therapists were randomized to receive either a financial or social incentive if they achieved a predetermined criterion on adherence to cognitive-behavioral therapy. In the first intervention period (block 1; 2 weeks), therapists received the reward they were initially randomized to if they achieved criterion. In the second intervention period (block 2; 2 weeks), therapists received both rewards if they achieved criterion. Therapists recorded 41 sessions across 15 unique clients over the project period. Primary outcomes included feasibility and acceptability. Feasibility was assessed quantitatively. Fifteen semi-structured interviews were conducted with therapists and leaders to assess acceptability. Difference in therapist adherence by condition was examined as an exploratory outcome. Adherence ratings were ascertained using an established and validated observational coding system of cognitive-behavioral therapy. RESULTS Both implementation strategies were feasible and acceptable-however, modifications to study design for the larger trial will be necessary based on participant feedback. With respect to our exploratory analysis, we found a trend suggesting the financial reward may have had a more robust effect on therapist adherence than the social reward. CONCLUSIONS Incentive-based implementation strategies can be feasibly administered in community mental health agencies with good acceptability, although iterative pilot work is essential. Larger, fully powered trials are needed to compare the effectiveness of implementation strategies to incentivize and enhance therapists' adherence to evidence-based practices such as cognitive-behavioral therapy.
Collapse
Affiliation(s)
- Rinad S Beidas
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA.
| | - Emily M Becker-Haimes
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA
| | - Danielle R Adams
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA
- School of Social Service Administration, University of Chicago, Chicago, USA
| | - Laura Skriner
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA
- New York-Presbyterian Hospital, Weill Cornell School of Medicine, New York, USA
| | - Rebecca E Stewart
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA
| | - Courtney Benjamin Wolk
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA
| | - Alison M Buttenheim
- School of Nursing, University of Pennsylvania, Philadelphia, USA
- The Center for Health Incentives and Behavioral Economics, University of Pennsylvania, Philadelphia, USA
| | | | | | | | - Steven C Marcus
- School of Social Policy and Practice, University of Pennsylvania, Philadelphia, USA
| |
Collapse
|