Improving Comparative Effectiveness Research of Complex Health Interventions: Standards from the Patient-Centered Outcomes Research Institute (PCORI)

Machine Learning Differentiation of Autism Spectrum Sub-Classifications

PURPOSE

Disorders on the autism spectrum have characteristics that can manifest as difficulties with communication, executive functioning, daily living, and more. These challenges can be mitigated with early identification. However, diagnostic criteria has changed from DSM-IV to DSM-5, which can make diagnosing a disorder on the autism spectrum complex. We evaluated machine learning to classify individuals as having one of three disorders of the autism spectrum under DSM-IV, or as non-spectrum.

METHODS

We employed machine learning to analyze retrospective data from 38,560 individuals. Inputs encompassed clinical, demographic, and assessment data.

RESULTS

The algorithm achieved AUROCs ranging from 0.863 to 0.980. The model correctly classified 80.5% individuals; 12.6% of individuals from this dataset were misclassified with another disorder on the autism spectrum.

CONCLUSION

Machine learning can classify individuals as having a disorder on the autism spectrum or as non-spectrum using minimal data inputs.

Testing effectiveness and implementation of a standardized approach to sexual dysfunction screening among adolescent and young adult-aged survivors of childhood cancer: A type I hybrid, mixed methods trial protocol

BACKGROUND

Approximately 20-50% of adolescent and young adult-aged childhood cancer survivors (AYA-CCS) experience sexual dysfunction (SD), although this healthcare need is widely underrecognized. Previous research from both AYA-CCS patients and their providers report that SD needs are unaddressed despite patient desires for SD discussions to be incorporated as part of their care. Patients and providers agree that standardized use of a patient-reported outcome measure may facilitate SD discussions; an SD screening approach was developed with patient and provider input. This study will measure the effectiveness of a standardized SD screening intervention and assess implementation outcomes and multilevel barriers and facilitators to guide future research.

METHODS

This multi-site, mixed methods, type 1 effectiveness-implementation hybrid trial will be evaluated using a pre-post design (NCT05524610). The trial will enroll 86 AYA-CCS (ages 15-39) from two cancer centers in the United States. The SD intervention consists of core fundamental functions with a "menu" of intervention options to allow for flexibility in delivery and tailoring in variable contexts. Effectiveness of the intervention on facilitating SD communication will be measured through patient surveys and clinical data; multivariable logistic regression will be used for the binary outcome of self-reported SD screening, controlling for patient-level predictors. Implementation outcomes will be assessed using mixed methods (electronic health record abstraction, patient and provider surveys, and provider interviews. Quantitative and qualitative findings will be merged using a joint display to understand factors affecting intervention success.

IMPLICATIONS

Identification and treatment of SD in AYA-CCS is an important and challenging quality of life concern. The type 1 hybrid design will facilitate rapid translation from research to practice by testing the effects of the intervention while simultaneously identifying multilevel barriers and facilitators to real-world implementation. This approach will inform future testing and dissemination of the SD screening intervention.

Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials

Pragmatism in clinical trials is focused on increasing the generalizability of research findings for routine clinical care settings. Hybridism in clinical trials (i.e., assessing both clinical effectiveness and implementation success) is focused on speeding up the process by which evidence-based practices are developed and adopted into routine clinical care. Even though pragmatic trial methodologies and implementation science evolved from very different disciplines, Pragmatic Trials and Hybrid Effectiveness-Implementation Trials share many similar design features. In fact, these types of trials can easily be conflated, creating the potential for investigators to mislabel their trial type or mistakenly use the wrong trial type to answer their research question. Blurred boundaries between trial types can hamper the evaluation of grant applications, the scientific interpretation of findings, and policy-making. Acknowledging that most trials are not pure Pragmatic Trials nor pure Hybrid Effectiveness-Implementation Trials, there are key differences in these trial types and they answer very different research questions. The purpose of this paper is to clarify the similarities and differences of these trial types for funders, researchers, and policy-makers. In addition, recommendations are offered to help investigators choose, label, and operationalize the most appropriate trial type to answer their research question. These recommendations complement existing reporting guidelines for clinical effectiveness trials (TIDieR) and implementation trials (StaRI).

Systematic review of contemporary interventions for improving discharge support and transitions of care from the patient experience perspective

AIM

To synthesize the impact of improvement interventions related to care coordination, discharge support and care transitions on patient experience measures.

METHOD

Systematic review. Searches were completed in six scientific databases, five specialty journals, and through snowballing. Eligibility included studies published in English (2015-2023) focused on improving care coordination, discharge support, or transitional care assessed by standardized patient experience measures as a primary outcome. Two independent reviewers made eligibility decisions and performed quality appraisals.

RESULTS

Of 1240 papers initially screened, 16 were included. Seven studies focused on care coordination activities, including three randomized controlled trials [RCTs]. These studies used enhanced supports such as improvement coaching or tailoring for vulnerable populations within Patient-Centered Medical Homes or other primary care sites. Intervention effectiveness was mixed or neutral relative to standard or models of care or simpler supports (e.g., improvement tool). Eight studies, including three RCTs, focused on enhanced discharge support, including patient education (e.g., teach back) and telephone follow-up; mixed or neutral results on the patient experience were also found and with more substantive risks of bias. One pragmatic trial on a transitional care intervention, using a navigator support, found significant changes only for the subset of uninsured patients and in one patient experience outcome, and had challenges with implementation fidelity.

CONCLUSION

Enhanced supports for improving care coordination, discharge education, and post-discharge follow-up had mixed or neutral effectiveness for improving the patient experience with care, compared to standard care or simpler improvement approaches. There is a need to advance the body of evidence on how to improve the patient experience with discharge support and transitional approaches.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

The influence of bias in randomized controlled trials on rehabilitation intervention effect estimates: what we have learned from meta-epidemiological studies

This study aimed to synthesize evidence from studies that addressed the influence of bias domains in randomized controlled trials on rehabilitation intervention effect estimates and discuss how these findings can maximize the trustworthiness of an RCT in rehabilitation. We screened studies about the influence of bias on rehabilitation intervention effect estimates published until June 2023. The characteristics and results of the included studies were categorized based on methodological characteristics and summarized narratively. We included seven studies with data on 227,806 RCT participants. Our findings showed that rehabilitation intervention effect estimates are likely exaggerated in trials with inadequate/unclear sequence generation and allocation concealment when using continuous outcomes. The influence of blinding was inconsistent and different from the rest of medical science, as meta-epidemiological studies showed overestimation, underestimation, or neutral associations for different types of blinding on rehabilitation treatment effect estimates. Still, it showed a more consistent pattern when looking at patient-reported outcomes. The impact of attrition bias and intention to treat has been analyzed only in two studies with inconsistent results. The risk of reporting bias seems to be associated with overestimation of treatment effects. Bias domains can influence rehabilitation treatment effects in different directions. The evidence is mixed and inconclusive due to the poor methodological quality of RCTs and the limited number and quality of studies looking at the influence of bias and treatment effects in rehabilitation. Further studies about the influence of bias in RCTs on rehabilitation intervention effect estimates are needed.

Design of a dual randomized trial in a type 2 hybrid effectiveness-implementation study

BACKGROUND

Dual randomized controlled trials (DRCT) are type 2 hybrid studies that include two randomized trials: one testing implementation strategies and one testing an intervention. We argue that this study design offers efficiency by providing rigorous investigation of both implementation and intervention in one study and has potential to accelerate generation of the evidence needed to translate interventions that work into real-world practice. Nevertheless, studies using this design are rare in the literature.

MAIN TEXT

We construct a paradigm that breaks down the components of the DRCT and provide a step-by-step explanation of features of the design and recommendations for use. A clear distinction is made between the dual strands that test the implementation versus the intervention, and a minimum of three randomized arms is advocated. We suggest an active treatment arm that includes both the implementation strategy and intervention that are hypothesized to be superior. We suggest two comparison/control arms: one to test the implementation strategy and the second to test the intervention. Further, we recommend selection criteria for the two control arms that place emphasis on maximizing the utility of the study design to advance public health practice.

CONCLUSIONS

On the surface, the design of a DRCT can appear simple, but actual application is complex. We believe it is that complexity that has limited its use in the literature. We hope that this paper will give both implementation scientists and trialists who are not familiar with implementation science a better understanding of the DRCT design and encouragement to use it.

A Systematic Review of Intervention Studies That Address HIV-Related Stigmas Among US Healthcare Workers and Health Systems: Applying a Theory-Based Ontology to Link Intervention Types, Techniques, and Mechanisms of Action to Potential Effectiveness

BACKGROUND

To end the HIV epidemic, we need to better understand how to address HIV-related stigmas in healthcare settings, specifically the common theoretical bases across interventions so that we can generalize about their potential effectiveness.

PURPOSE

We describe theory-based components of stigma interventions by identifying their functions/types, techniques, and purported mechanisms of change.

METHODS

This systematic review examined studies published by April 2021. We applied a transtheoretical ontology developed by the Human Behaviour Change Project, consisting of 9 intervention types (ITs), 93 behavior change techniques (BCTs), and 26 mechanisms of action (MOAs). We coded the frequency and calculated the potential effectiveness of each IT, BCT, and MOA. We evaluated study quality with a 10-item adapted tool.

RESULTS

Among the nine highest quality studies, indicated by the use of an experimental design, the highest potentially effective IT was "Persuasion" (i.e. using communication to induce emotions and/or stimulate action; 66.7%, 4/6 studies). The highest potentially effective BCTs were "Behavioral practice/rehearsal" (i.e. to increase habit and skill) and "Salience of consequences" (i.e. to make consequences of behavior more memorable; each 100%, 3/3 studies). The highest potentially effective MOAs were "Knowledge" (i.e. awareness) and "Beliefs about capabilities" (i.e. self-efficacy; each 67%, 2/3 studies).

CONCLUSIONS

By applying a behavior change ontology across studies, we synthesized theory-based findings on stigma interventions. Interventions typically combined more than one IT, BCT, and MOA. Practitioners and researchers can use our findings to better understand and select theory-based components of interventions, including areas for further evaluation, to expedite ending the HIV epidemic.

Coordinated Health Care Interventions for Childhood Asthma Gaps in Outcomes (CHICAGO) plan

Background

Evidence-based strategies to improve outcomes in minority children with uncontrolled asthma discharged from the emergency department (ED) are needed.

Objectives

This multicenter pragmatic clinical trial was designed to compare an ED-only intervention (decision support tool), an ED-only intervention and home visits by community health workers for 6 months (ED-plus-home), and enhanced usual care (UC).

Methods

Children aged 5 to 11 years with uncontrolled asthma were enrolled. The change over 6 months in the Patient-Reported Outcomes Measurement Information System Asthma Impact Scale score in children and Satisfaction with Participation in Social Roles score in caregivers were the primary outcomes. The secondary outcomes included guideline-recommended ED discharge care and self-management.

Results

Recruitment was significantly lower than expected (373 vs 640 expected). Of the 373 children (64% Black and 31% Latino children), only 63% completed the 6-month follow-up visit. In multivariable analyses that accounted for missing data, the adjusted odds ratios and 98% CIs for differences in Asthma Impact Scores or caregivers' Satisfaction with Participation in Social Roles scores were not significant. However, guideline-recommended ED discharge care was significantly improved in the intervention groups versus in the UC group, and self-management behaviors were significantly improved in the ED-plus-home group versus in the ED-only and UC groups.

Conclusions

The ED-based interventions did not significantly improve the primary clinical outcomes, although the study was likely underpowered. Although guideline-recommended ED discharge care and self-management did improve, their effect on clinical outcomes needs further study.

An external facilitation intervention to increase uptake of an adverse drug event reporting intervention

Background

Adverse drug events (ADEs) are a leading cause of emergency department visits and hospital admissions in Canada. ActionADE prevents repeat ADEs by enabling clinicians to document and communicate standardized ADE information across care settings. We used an external facilitation intervention to promote the uptake of ActionADE in four hospitals in British Columbia, Canada. This study examined whether, how and in what context external facilitation influenced the uptake of ActionADE.

Methods

In this convergent-parallel mixed-methods study, an external facilitator used a four-step iterative process to support site champions using context-specific implementation strategies to increase the ADE reporting rate at their sites. We extracted archival data to assess implementation determinants before and after the implementation of the external facilitation and implementation strategies. We also retrieved data on the mean monthly counts of reported ADEs for each user from the ActionADE server. Zero-inflated Poisson models were used to examine changes in mean monthly counts of reported ADEs per user between pre-intervention (June 2021 to October 2021) and intervention (November 2021 to March 2022) periods.

Results

The external facilitator and site champions co-created three functions: (1) educate pharmacists about what and how to report in ActionADE, (2) educate pharmacists about the impact of ActionADE on patient outcomes, and (3) provide social support for pharmacists to integrate ADE reporting into clinical workflows. Site champions used eight forms to address the three functions. Peer support and reporting competition were the two common strategies used by all sites. Sites' responses to external facilitation varied. The rate of mean monthly counts of reported ADEs per user significantly increased during the intervention period compared to the pre-intervention period at LGH (RR: 3.74, 95% CI 2.78 to 5.01) and RH (RR: 1.43, 95% CI 1.23 to 1.94), but did not change at SPH (RR: 0.68, 95% CI: 0.43 to 1.09) and VGH (RR: 1.17, 95% CI 0.92 to 1.49). Leave of absence of the clinical pharmacist champion and failure to address all identified functions were implementation determinants that influenced the effectiveness of external facilitation.

Conclusion

External facilitation effectively supported researchers and stakeholders to co-create context-specific implementation strategies. It increased ADE reporting at sites where clinical pharmacist champions were available, and where all functions were addressed.

From guidelines to decision aids and adherence supports: Insights from the process of evidence translation

OBJECTIVE

To explore the evidence-translator's experience of the expert-recommended process of translating guidelines into tools for decision making, action, and adherence with the goal of improvement.

METHODS

A single reviewer dual reviewed the content, quality, certainty, and applicability of primary atherosclerotic cardiovascular prevention guidelines from the U.S. Preventive Services Task Force at the time of this work and used targeted searches of Medline to define the ideal structure and outcomes of tools; fill in gaps in guidelines; identify end-user needs; and choose and optimize existing tools in preparation for testing.

RESULTS

Guidelines addressed screening, treatments, and/or supports, but never the combination of all three. None provided all of the information needed for evidence translation. Searches in Medline filled in some evidence gaps and provided key insights into end-user needs and effective tools. However, evidence translators are left with complicated decisions about how to use and align evidence.

CONCLUSION

Guidelines provide some, but not all, of the evidence needed for evidence translation, requiring additional intensive work. Evidence gaps result in complicated decisions about how to use and align evidence and balance feasibility and rigor.

PRACTICE IMPLICATIONS

Guidelines, standards groups, and researchers should work to better support the process of evidence translation.

Intervention Fidelity in Pain Pragmatic Trials for Nonpharmacologic Pain Management: Nuanced Considerations for Determining PRECIS-2 Flexibility in Delivery and Adherence

Nonpharmacological treatments are considered first-line pain management strategies, but they remain clinically underused. For years, pain-focused pragmatic clinical trials (PCTs) have generated evidence for the enhanced use of nonpharmacological interventions in routine clinical settings to help overcome implementation barriers. The Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) framework describes the degree of pragmatism across 9 key domains. Among these, "flexibility in delivery" and "flexibility in adherence," address a key goal of pragmatic research by tailoring approaches to settings in which people receive routine care. However, to maintain scientific and ethical rigor, PCTs must ensure that flexibility features do not compromise delivery of interventions as designed, such that the results are ethically and scientifically sound. Key principles of achieving this balance include clear definitions of intervention core components, intervention monitoring and documentation that is sufficient but not overly burdensome, provider training that meets the demands of delivering an intervention in real-world settings, and use of an ethical lens to recognize and avoid potential trial futility when necessary and appropriate. PERSPECTIVE: This article presents nuances to be considered when applying the PRECIS-2 framework to describe pragmatic clinical trials. Trials must ensure that patient-centered treatment flexibility does not compromise delivery of interventions as designed, such that measurement and analysis of treatment effects is reliable.

Integrating Alcohol-Related Prevention and Treatment Into Primary Care: A Cluster Randomized Implementation Trial

Importance

Unhealthy alcohol use is common and affects morbidity and mortality but is often neglected in medical settings, despite guidelines for both prevention and treatment.

Objective

To test an implementation intervention to increase (1) population-based alcohol-related prevention with brief interventions and (2) treatment of alcohol use disorder (AUD) in primary care implemented with a broader program of behavioral health integration.

Design, Setting, and Participants

The Sustained Patient-Centered Alcohol-Related Care (SPARC) trial was a stepped-wedge cluster randomized implementation trial, including 22 primary care practices in an integrated health system in Washington state. Participants consisted of all adult patients (aged ≥18 years) with primary care visits from January 2015 to July 2018. Data were analyzed from August 2018 to March 2021.

Interventions

The implementation intervention included 3 strategies: practice facilitation; electronic health record decision support; and performance feedback. Practices were randomly assigned launch dates, which placed them in 1 of 7 waves and defined the start of the practice's intervention period.

Main Outcomes and Measures

Coprimary outcomes for prevention and AUD treatment were (1) the proportion of patients who had unhealthy alcohol use and brief intervention documented in the electronic health record (brief intervention) for prevention and (2) the proportion of patients who had newly diagnosed AUD and engaged in AUD treatment (AUD treatment engagement). Analyses compared monthly rates of primary and intermediate outcomes (eg, screening, diagnosis, treatment initiation) among all patients who visited primary care during usual care and intervention periods using mixed-effects regression.

Results

A total of 333 596 patients visited primary care (mean [SD] age, 48 [18] years; 193 583 [58%] female; 234 764 [70%] White individuals). The proportion with brief intervention was higher during SPARC intervention than usual care periods (57 vs 11 per 10 000 patients per month; P < .001). The proportion with AUD treatment engagement did not differ during intervention and usual care (1.4 vs 1.8 per 10 000 patients; P = .30). The intervention increased intermediate outcomes: screening (83.2% vs 20.8%; P < .001), new AUD diagnosis (33.8 vs 28.8 per 10 000; P = .003), and treatment initiation (7.8 vs 6.2 per 10 000; P = .04).

Conclusions and Relevance

In this stepped-wedge cluster randomized implementation trial, the SPARC intervention resulted in modest increases in prevention (brief intervention) but not AUD treatment engagement in primary care, despite important increases in screening, new diagnoses, and treatment initiation.

Trial Registration

ClinicalTrials.gov Identifier: NCT02675777.

Characterizing modifications to a comparative effectiveness research study: the OPTIMIZE trial-using the Framework for Reporting Adaptations and Modifications to Evidence-based Interventions (FRAME)

BACKGROUND

The OPTIMIZE trial is a multi-site, comparative effectiveness research (CER) study that uses a Sequential Multiple Assessment Randomized Trial (SMART) designed to examine the effectiveness of complex health interventions (cognitive behavioral therapy, physical therapy, and mindfulness) for adults with chronic low back pain. Modifications are anticipated when implementing complex interventions in CER. Disruptions due to COVID have created unanticipated challenges also requiring modifications. Recent methodologic standards for CER studies emphasize that fully characterizing modifications made is necessary to interpret and implement trial results. The purpose of this paper is to outline the modifications made to the OPTIMIZE trial using the Framework for Reporting Adaptations and Modifications to Evidence-Based Interventions (FRAME) to characterize modifications to the OPTIMIZE trial in response to the COVID pandemic and other challenges encountered.

METHODS

The FRAME outlines a strategy to identify and report modifications to evidence-based interventions or implementation strategies, whether planned or unplanned. We use the FRAME to characterize the process used to modify the aspects of the OPTIMIZE trial. Modifications were made to improve lower-than-anticipated rates of treatment initiation and COVID-related restrictions. Contextual modifications were made to permit telehealth delivery of treatments originally designed for in-person delivery. Training modifications were made with study personnel to provide more detailed information to potential participants, use motivational interviewing communication techniques to clarify potential participants' motivation and possible barriers to initiating treatment, and provide greater assistance with scheduling of assigned treatments.

RESULTS

Modifications were developed with input from the trial's patient and stakeholder advisory panels. The goals of the modifications were to improve trial feasibility without compromising the interventions' core functions. Modifications were approved by the study funder and the trial steering committee.

CONCLUSIONS

Full and transparent reporting of modifications to clinical trials, whether planned or unplanned, is critical for interpreting the trial's eventual results and considering future implementation efforts.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03859713. Registered on March 1, 2019.

Pragmatic Clinical Trial to Improve Patient Experience Among Adults During Transitions from Hospital to Home: the PArTNER study

INTRODUCTION

Minority-serving hospitals (MSHs) need evidence-based strategies tailored to the populations they serve to improve patient-centered outcomes after hospitalization.

METHODS

We conducted a pragmatic randomized clinical trial (RCT) from October 2014 to January 2017 at a MSH comparing the effectiveness of a stakeholder-supported Navigator intervention vs. Usual care on post-hospital patient experience, outcomes, and healthcare utilization. Community health workers and peer coaches delivered the intervention which included (1) in-hospital visits to assess barriers to health/healthcare and to develop a personalized Discharge Patient Education Tool (DPET); (2) a home visit to review the DPET; and (3) telephone-based peer coaching. The co-primary outcomes were between-group comparisons of 30-day changes in Patient-Reported Outcomes Measurement Information System (PROMIS) measures of anxiety and informational support (minimum important difference is 2 to 5 units change); a p-value <0.025 was considered significant using intention-to-treat analysis. Secondary outcomes included death, ED visits, or readmissions and measures of emotional, social, and physical health at 30 and 60 days.

RESULTS

We enrolled 1029 adults hospitalized with heart failure (28%), pneumonia (22%), MI (10%), COPD (11%), or sickle cell disease (29%). Over 80% were non-Hispanic Black. Overall, there were no significant between-group differences in the 30-day change in anxiety (adjusted difference: -1.6, 97.5% CI -3.3 to 0.1, p=0.03), informational support (adjusted difference: -0.01, 97.5% CI -2.0 to 1.9, p=0.99), or any secondary outcomes. Exploratory analyses suggested the Navigator intervention improved anxiety among participants with COPD, a primary care provider, a hospitalization in the past 12 months, or higher baseline anxiety; among participants without health insurance, the intervention improved informational support (all p-values <0.05).

CONCLUSIONS

In this pragmatic RCT at a MSH, the Navigator intervention did not improve post-hospital anxiety, informational support, or other outcomes compared to Usual care. Benefits observed in participant subgroups should be confirmed in future studies.

TRIAL REGISTRATION

Clinicaltrials.gov identifier: NCT02114515.

Designing Interventions Addressing Structural Racism to Reduce Kidney Health Disparities: A Report from a National Institute of Diabetes and Digestive and Kidney Diseases Workshop

Structural racism embodies the many ways in which society fosters racial discrimination through "mutually reinforcing inequitable systems" that limit access to resources and opportunities that can promote health and well being among marginalized communities. To achieve health equity, and kidney health equity more specifically, structural racism must be eliminated. In February 2022, the National Institute of Diabetes and Digestive and Kidney Diseases convened the "Designing Interventions that Address Structural Racism to Reduce Kidney Health Disparities" workshop, which was aimed at describing the mechanisms through which structural racism contributes to health and health care disparities for people along the continuum of kidney disease and identifying actionable opportunities for interventional research focused on dismantling or addressing the effects of structural racism. Participants identified six domains as key targets for interventions and future research: (1) apply an antiracism lens, (2) promote structural interventions, (3) target multiple levels, (4) promote effective community and stakeholder engagement, (5) improve data collection, and (6) advance health equity through new health care models. There is an urgent need for research to develop, implement, and evaluate interventions that address the unjust systems, policies, and laws that generate and perpetuate inequities in kidney health.

Balancing adaptability and standardisation: insights from 27 routinely implemented ICHOM standard sets

BACKGROUND

Healthcare systems around the world experience increasing pressure to control future growth of healthcare expenditures. Among other initiatives, quality and value-based benchmarking has become an important field to inform clinical evaluation and reimbursement questions. The International Consortium for Health Outcomes Measurement (ICHOM) has become one of the driving forces to translate scientific evidence into standardized assessments that are routinely applicable in day-to-day care settings. These aim to provide a benchmarking tool that allows the comparison and competition of health care delivery on the basis of value-based health care principles.

METHODS

This work focuses on the consolidation of the ICHOM methodology and presents insights from 27 routinely implemented Standard Sets. The analysis is based on a literature review of the ICHOM literature repository, a process document review and key informant interviews with ICHOM's outcomes research and development team.

RESULTS

Key findings are that the scope of ICHOM Standard Sets shifted from a more static focus on burden of disease and poorly standardized care pathways to a more dynamic approach that also takes into account questions about the setting of care, feasibility of implementing a benchmarking tool and compatibility of different Standard Sets. Although certain overlaps exist with other initiatives in the field of patient reported outcomes (PRO), their scopes differ significantly and they hence rather complement each other. ICHOM pursues a pragmatic approach to enable the benchmarking and the analysis of healthcare delivery following the principles of value-based healthcare.

CONCLUSION

The ICHOM Standard Sets complement other initiatives in the field of patient-reported outcomes (PRO) and functional reporting by placing a particular focus on healthcare delivery, while other initiatives primarily focus on evaluation of academic endpoints. Although ICHOM promotes a pragmatic approach towards developing and devising its Standard Sets, the definition of standardized decision making processes emerged as one of the key challenges. Furthermore, the consolidation of core metrics across number of disease areas to enable the parallel implementation of different Standard Sets in the same care setting is an important goal that will enable the widespread implementation of patient-reported outcome measures (PROM).

BeatPain Utah: study protocol for a pragmatic randomised trial examining telehealth strategies to provide non-pharmacologic pain care for persons with chronic low back pain receiving care in federally qualified health centers

INTRODUCTION

Although evidence-based guidelines recommend non-pharmacologic treatments as first-line care for chronic low back pain (LBP), uptake has been limited, particularly in rural, low-income and ethnically diverse communities. The BeatPain study will evaluate the implementation and compare the effectiveness of two strategies to provide non-pharmacologic treatment for chronic LBP. The study will use telehealth to overcome access barriers for persons receiving care in federally qualified health centres (FQHCs) in the state of Utah.

METHODS AND ANALYSIS

BeatPain Utah is a pragmatic randomised clinical trial with a hybrid type I design investigating different strategies to provide non-pharmacologic care for adults with chronic LBP seen in Utah FQHCs. The intervention strategies include a brief pain consult (BPC) and telehealth physical therapy (PT) component provided using either an adaptive or sequenced delivery strategy across two 12-week treatment phases. Interventions are provided via telehealth by centrally located physical therapists. The sequenced delivery strategy provides the BPC, followed by telehealth PT in the first 12 weeks for all patients. The adaptive strategy uses a stepped care approach and provides the BPC in the first 12 weeks and telehealth PT to patients who are non-responders to the BPC component. We will recruit 500 English-speaking or Spanish-speaking participants who will be individually randomised with 1:1 allocation. The primary outcome is the Pain, Enjoyment and General Activity measure of pain impact with secondary outcomes including the additional pain assessment domains specified by the National Institutes (NIH) of Health Helping to End Addiction Long Initiative and implementation measures. Analyses of primary and secondary measures of effectiveness will be performed under longitudinal mixed effect models across assessments at baseline, and at 12, 26 and 52 weeks follow-ups.

ETHICS AND DISSEMINATION

Ethics approval for the study was obtained from the University of Utah Institutional Review Board. On completion, study data will be made available in compliance with NIH data sharing policies.

TRIAL REGISTRATION NUMBER

NCT04923334.

Effects of goal-oriented care for adults with multimorbidity: A systematic review and meta-analysis

OBJECTIVE

To systematically review the evidence from randomized controlled trials comparing the effects of goal-oriented care against standard care for multimorbid adults.

DATA SOURCES/STUDY SETTING

The literature presenting the results of randomized trials assessing the outcomes of goal-oriented care compared with usual care for adults with multimorbidity.

STUDY DESIGN

Systematic review and meta-analysis.

DATA COLLECTION/EXTRACTION METHODS

We searched the Cochrane Database of Systematic Reviews (CENTRAL), EMBASE, MEDLINE, CINHAL, trial registries such as ClinicalTrial.gov and World Health Organization International Clinical Trials Registry Platform (ICTRP), and the references of eligible trials and relevant reviews. Goal-oriented care was defined as an approach that engages patients, establishes personal goals, and sets targets for patients and clinicians to plan a course of action and measure outcome. We reviewed 228 trials, and 12 were included. We extracted outcome data on quality of life, hospital admission, patients' satisfaction, patient and caregiver burden. Risk of bias was assessed and certainty of evidence was evaluated using GRADE.

PRINCIPAL FINDINGS

No study was fully free of bias. No effect was found on quality of life (standardized mean difference [SMD]: 0.05; 95% CI: -0.05 to 0.16) and hospital admission (risk ratio [RR]: 0.87; 95% CI: 0.65 to 1.17). There was a very small effect for patients' satisfaction (SMD: 0.15; 95% CI: 0.00 to 0.29) and caregiver burden (SMD: -0.13; 95% CI: -0.26 to 0.00). Certainty of evidence was low for all outcomes.

CONCLUSIONS

No firm conclusions can be reached about the effects of goal-oriented care for multimorbid adults. Future research should overcome the shortcomings of trials assessed in this meta-analysis. Sound application of the indications for research of complex healthcare interventions is warranted.

Co-creating the Patient Partner Guide by a Multiple Chronic Conditions Team of Patients, Clinicians, and Researchers: Observational Report

BACKGROUND

Engaging patients as partners can influence research, with rewards and deterrents. The authors are researchers and patient co-investigators who collaborated on a comparative effectiveness, randomized controlled study of a structured quality improvement (QI) process to improve behavioral health and primary care integration for people managing multiple chronic conditions (MCC). Patient co-investigators responded to a gap in available resources to support study clinics in partnering with their own patients in QI and co-created the Patient Partner Guide (PPG).

OBJECTIVE

Describe the development of the PPG, its use by clinics undertaking the QI project, and research team partnerships.

DESIGN

Observational report of study intervention component.

PARTICIPANTS

Diverse patients and family members managing MCC and members of their primary care clinics.

INTERVENTION

The PPG component of the study intervention is a five-step workbook providing practical tools and resources to sustain partnerships across clinic QI team members, including patient partners. The process of developing the PPG relied on relationship-building tools that were iteratively assessed, practiced, improved, and incorporated into the PPG under the leadership of patient co-investigators.

MAIN MEASURES

Observations related to PPG use and patient partner inclusion in clinic QI; impact on the research team.

KEY RESULTS

Of 20 clinics, 6 engaged patients as full partners on QI teams. Clinics found resistance in partnering and challenges in using the PPG but valued the material and their partners' contributions. Similarly, engagement of patient co-investigators in research brought a shift in perspective to team members. The PPG is available and was adapted for use by research teams.

CONCLUSIONS

Engagement of patients and other stakeholders in research can be transformative and productive. Building relationships through meaningful work benefits others, and in turn, the research process. This approach can enhance clinical care QI and may result in substantial contributions to the conduct of research.

CLINICAL TRIAL REGISTRATION

https://clinicaltrials.gov/ct2/show/NCT02868983.

Use of Theoretical Frameworks in the Development and Testing of Palliative Care Interventions

CONTEXT

Palliative care (PC) research has grown over the last 20 years. Yet, the causal components and pathways of PC interventions remain unclear.

OBJECTIVES

To document the prevalence and application of theoretical frameworks in developing and testing PC interventions.

METHODS

We conducted a secondary analysis of previously published systematic reviews of PC randomized clinical trials. Trials were evaluated for explicit mention of a theoretical framework, process or delivery model, or clinical practice guideline that supported the development of the intervention. We used a structured data extraction form to document study population, outcomes, and whether and how authors used a theoretical framework, process/delivery model, or clinical practice guideline. We applied an adapted coding scheme to evaluate use of theoretical frameworks.

RESULTS

We reviewed 85 PC trials conducted between 1984 and 2021. Thirty-eight percent (n = 32) of trials explicitly mentioned a theoretical framework, process or delivery model, or clinical practice guideline as a foundation for the intervention design. Only nine trials included a theoretical framework, while the remaining 23 cited a process/delivery model or clinical practice guideline.

CONCLUSION

Most PC trials do not cite a theoretical foundation for their intervention design. Future work should focus on developing and validating new theoretical frameworks and modifying existing theories and models to better explain the mechanisms of the variety of PC interventions.

Guidelines for artificial intelligence in medicine: A literature review and content analysis of frameworks (Preprint)

Background

Artificial intelligence (AI) is rapidly expanding in medicine despite a lack of consensus on its application and evaluation.

Objective

We sought to identify current frameworks guiding the application and evaluation of AI for predictive analytics in medicine and to describe the content of these frameworks. We also assessed what stages along the AI translational spectrum (ie, AI development, reporting, evaluation, implementation, and surveillance) the content of each framework has been discussed.

Methods

We performed a literature review of frameworks regarding the oversight of AI in medicine. The search included key topics such as “artificial intelligence,” “machine learning,” “guidance as topic,” and “translational science,” and spanned the time period 2014-2022. Documents were included if they provided generalizable guidance regarding the use or evaluation of AI in medicine. Included frameworks are summarized descriptively and were subjected to content analysis. A novel evaluation matrix was developed and applied to appraise the frameworks’ coverage of content areas across translational stages.

Results

Fourteen frameworks are featured in the review, including six frameworks that provide descriptive guidance and eight that provide reporting checklists for medical applications of AI. Content analysis revealed five considerations related to the oversight of AI in medicine across frameworks: transparency, reproducibility, ethics, effectiveness, and engagement. All frameworks include discussions regarding transparency, reproducibility, ethics, and effectiveness, while only half of the frameworks discuss engagement. The evaluation matrix revealed that frameworks were most likely to report AI considerations for the translational stage of development and were least likely to report considerations for the translational stage of surveillance.

Conclusions

Existing frameworks for the application and evaluation of AI in medicine notably offer less input on the role of engagement in oversight and regarding the translational stage of surveillance. Identifying and optimizing strategies for engagement are essential to ensure that AI can meaningfully benefit patients and other end users.

Walking the talk on multi-level interventions: The power of parsimony

There is strong consensus regarding the need for multi-level interventions (MLIs) to address today's complex health problems. Several longstanding social ecological frameworks are commonly referred to in guiding MLI development. The specificity and comprehensiveness of these frameworks unwittingly suggest that the totality of included influences are important in all health contexts. Not surprisingly, when viewed as requiring intervention at all levels of influence, MLIs are often considered to be infeasible due to sizeable cost and logistical barriers. Thus, efforts to develop and evaluate MLIs have been extremely limited, and comparatively few examples are found in the health literature. We argue that operational frameworks to identify which levels matter in which contexts - henceforth, referred to as parsimony - could accelerate the field towards broader use of MLIs. We suggest a hypothetical operational framework informed by complexity theory and pragmatic approaches that could enable us to conceptualize, design and evaluate MLIs to consider where reflexive and recursive process mechanisms that cross levels should be targeted by MLI. The approach also emphasizes sustainability of MLIs. Without developing parsimony-based operational frameworks to move us forward, we fear that little will change, and we will simply continue to talk, without proceeding to the walk.