Informing the development of the SUCCEED reporting guideline for studies on the scaling of health interventions: A systematic review

BACKGROUND

Quality reporting contributes to effective translation of health research in practice and policy. As an initial step in the development of a reporting guideline for scaling, the Standards for reporting stUdies of sCaling evidenCEd-informED interventions (SUCCEED), we performed a systematic review to identify relevant guidelines and compile a list of potential items.

METHODS

We conducted a systematic review according to Cochrane method guidelines. We searched the following databases: MEDLINE, Embase, PsycINFO, Cochrane Library, CINAHL, Web of Science, from their respective inceptions. We also searched websites of relevant organizations and Google. We included any document that provided instructions or recommendations, e.g., reporting guideline, checklist, guidance, framework, standard; could inform the design or reporting of scaling interventions; and related to the health sector. We extracted characteristics of the included guidelines and assessed their methodological quality using a 3-item internal validity assessment tool. We extracted all items from the guidelines and classified them according to the main sections of reporting guidelines (title, abstract, introduction, methods, results, discussion and other information). We performed a narrative synthesis based on descriptive statistics.

RESULTS

Of 7704 records screened (published between 1999 and 2019), we included 39 guidelines, from which data were extracted from 57 reports. Of the 39 guidelines, 17 were for designing scaling interventions and 22 for reporting implementation interventions. At least one female author was listed in 31 guidelines, and 21 first authors were female. None of the authors belonged to the patient stakeholder group. Only one guideline clearly identified a patient as having participated in the consensus process. More than half the guidelines (56%) had been developed using an evidence-based process. In total, 750 items were extracted from the 39 guidelines and distributed into the 7 main sections.

CONCLUSION

Relevant items identified could inform the development of a reporting guideline for scaling studies of evidence-based health interventions. This and our assessment of guidelines could contribute to better reporting in the science and practice of scaling.

A New Era for Firearm Violence Prevention Research

This Viewpoint discusses the expansion of firearm injury research that involves diverse disciplinary perspectives that could potentially lead to lifesaving policy innovation.

Asian American Pacific Islander Representation in Outcomes Research: NSQIP Scoping Review

BACKGROUND

If Asian American and Pacific Islanders (AAPIs) are not recognized within patients in health services research, we miss an opportunity to ensure health equity in patient outcomes. However, it is unknown what the rates are of AAPIs inclusion in surgical outcomes research.

STUDY DESIGN

Through a scoping review, we used Covidence to search MEDLINE, EMBASE, PsycINFO, Web of Science, Scopus, and CINAHL for studies published in 2008-2018 using NSQIP data. NSQIP was chosen because of its national scope, widespread use in research, and coding inclusive of AAPI patients. We examined the proportion of studies representing AAPI patients in the demographic characteristics and Methods, Results, or Discussion section. We then performed multivariable logistic regression to examine associations between study characteristics and AAPI inclusion.

RESULTS

In 1,264 studies included for review, 62% included race. Overall, only 22% (n = 278) of studies included AAPI patients. Of studies that included race, 35% represented AAPI patients in some component of the study. We found no association between sample size or publication year and inclusion. Studies were significantly more likely to represent AAPI patients when there was a higher AAPI population in the region of the first author's institution (lowest vs highest tercile; p < 0.001). Studies with a focus on disparities were more likely to include AAPI patients (p = 0.001).

CONCLUSIONS

Our study is the first to examine AAPI representation in surgical outcomes research. We found < 75% of studies examine race, despite availability within NSQIP. Little more than one-third of studies including race reported on AAPI patients as a separate group. To provide the best care, we must include AAPI patients in our research.

Realizing the Promise of Learning Organizations to Transform Mental Health Care: Telepsychiatry Care As an Exemplar

To address the global mental health crisis exacerbated by the COVID-19 pandemic, an urgent need has emerged to transform the accessibility, efficiency, and quality of mental health care. The next suite of efforts to transform mental health care must foster the implementation of "learning organizations," that is, organizations that continuously improve patient-centered care through ongoing data collection. The concept of learning organizations is highly regarded, but the key features of such organizations, particularly those providing mental health care, are less well defined. Using telepsychiatry care as an example, the authors of this Open Forum concretely describe the key building blocks for operationalizing a learning organization in mental health care to set a research agenda for services transformation.

Methodological standards for qualitative and mixed methods patient centered outcomes research

The Patient-Centered Outcomes Research Institute’s (PCORI) methodology standards for qualitative methods and mixed methods research help ensure that research studies are designed and conducted to generate the evidence needed to answer patients’ and clinicians’ questions about which methods work best, for whom, and under what circumstances. This set of standards focuses on factors pertinent to patient centered outcomes research, but it is also useful for providing guidance for other types of clinical research. The standards can be used to develop and evaluate proposals, conduct the research, and interpret findings. The standards were developed following a systematic process: survey the range of key methodological issues and potential standards, narrow inclusion to standards deemed most important, draft preliminary standards, solicit feedback from a content expert panel and the broader public, and use this feedback to develop final standards for review and adoption by PCORI’s board of governors. This article provides an example on how to apply the standards in the preparation of a research proposal.

Bystander Ethics and Good Samaritanism: A Paradox for Learning Health Organizations

In 2012, a U.S. Institute of Medicine report called for a different approach to health care: "Left unchanged, health care will continue to underperform; cause unnecessary harm; and strain national, state, and family budgets." The answer, they suggested, would be a "continuously learning" health system. Ethicists and researchers urged the creation of "learning health organizations" that would integrate knowledge from patient-care data to continuously improve the quality of care. Our experience with an ongoing research study on atrial fibrillation-a trial known as IMPACT-AFib-gave us some insight into one of the challenges that will have to be dealt with in creating these organizations. Although the proposed educational intervention study placed no restrictions on what providers and health plans could do, the oversight team argued that the ethical principle of beneficence did not allow the researchers to be "bystanders" in relation to a control group receiving suboptimal care. In response, the researchers designed a "workaround" that allowed the project to go forward. We believe the experience suggests that what we call "bystander ethics" will create challenges for the kinds of quality improvement research that LHOs are designed to do.

[Health services research in intensive care medicine in Germany : Status quo and future challenges exemplified by acute pulmonary failure]

Health services research (HSR) is a multidisciplinary field of research that describes disease treatment and health care and their framework conditions. In the last 20 years, the HSR aspect became more and more the clinical focus of intensive care medicine. Under this aspect HSR investigates the use of clinical measures and their impact on patient outcome under routine intensive care medical conditions. This article provides an overview of the current state of HSR in intensive care medicine in Germany using the example of acute respiratory distress syndrome (ARDS). The ARDS still represents a clinical disease with high intra-hospital mortality (30-60%) despite progress in intensive care medicine. Survivors of ARDS have substantial long-term limitations on physical and mental health. The treatment of ARDS patients is tedious, laborious for intensive care unit staff and complex. Despite evident treatment recommendations, these are only insufficiently implemented in the clinical routine. With the help of quality indicators, benchmarking, certification and peer review procedures, the quality of intensive care treatment in the clinical routine can be documented and improved. An important role in HSR is patient safety and focusing on the outcome with evaluation of the patient's will. As part of the establishment of the innovation fund for HSR, promising intensive medical care projects have been promoted to improve the quality of care and the quality of long-term outcome for intensive care patients. An important focus lies on the identification of factors that improve long-term quality of life after intensive care. The expansion of registries and telemedicine in intensive care offers the opportunity to bundle and share experiences more effectively and thereby establish (guideline-based) treatment recommendations faster in the clinical practice.

Implementation of Clinical Practice Guidelines in the Health Care Setting: A Concept Analysis

The literature is replete with clinical practice guidelines (CPGs) and evidence supporting them. Translating guidelines into practice, however, is often challenging. We conducted a concept analysis to define the concept of "implementation of CPGs in health care settings." We utilized Walker and Avant's methodology to define the concept of "implementation of CPGs in health care settings." This included a focused review of the literature, defining the relevant attributes, defining implementation, case examples, and antecedents and potential consequences from implementation of CPGs in health care settings. The concept "implementation" is complex, with numerous frameworks, facilitators, and barriers to implementation described in the literature. The existing literature supports our definition of implementation of CPGs in a health care setting as a process of changing practice in health care while utilizing the best level of evidence that is available in the published literature. These include 7 attributes necessary for effective implementation. Implementation of CPGs in health care settings requires an ongoing iterative process that considers these attributes and is inclusive to administrators, clinicians, and patients to ensure guidelines are understood, accepted, implemented, and evaluated for continued adoption of best practices. Ongoing efforts inclusive at all steps of implementation across multiple levels are needed to effectively change practice.

Designing the Future of Children's Mental Health Services

Advancements in evidence-based psychosocial interventions, digital technologies, and implementation strategies (i.e., health services research products) for youth mental health services have yet to yield significant improvement in public health outcomes. Achieving such impact will require that these research products are easy to use, useful, and contextually appropriate. This paper describes how human-centered design (HCD), an approach that aligns product development with the needs of the people and settings that use those products, can be leveraged to improve youth mental health services. We articulate how HCD can advance accessibility, effectiveness, and equity, with specific consideration of unique aspects of youth mental health services.

A proposed standardized approach to studying attrition in pediatric weight management

Pediatric obesity is a major public health issue. Lifestyle and behavioral interventions are the foundation of pediatric weight management; however, intervention effectiveness is compromised when families (children, youth, and/or parent[s]) discontinue care prematurely. Intervention attrition minimizes the potential health benefits derived from interventions, results in inefficient use of health services resources, and can magnify health disparities. Most attrition research in pediatric weight management has been descriptive, highlighting the need to advance the field, both academically and clinically. Herein, we propose a standard approach to studying attrition in pediatric weight management interventions to enhance our understanding, elevate the quality of research, enable study-to-study comparisons, and inform strategies designed to mitigate its impact. We focus on three issues. First, "Conceptualization and operationalization," whereby the processes underlying attrition from interventions should be decomposed into clinically important phases that are defined based on intervention characteristics. Relatedly, theoretically relevant variables should be identified with different mechanisms driving attrition in each phase. We propose a matrix of attrition, a tool designed to delineate the relevant stages of attrition and associated variables of analytical value. Second, "Pre-study" underscores the value of developing a plan to study attrition a priori rather than post hoc, including variable and sample size considerations, which broadens the range and quality of analysis. Finally, "Post-study" emphasizes comprehensive reporting of attrition, outlines typical comparisons of analytical interest, and statistical techniques used to handle missing data. Implications for clinical practice in pediatric weight management are discussed. Clinical Trial Registration: Not applicable.

Decision-based models of the implementation of interventions in systems of healthcare: Implementation outcomes and intervention effectiveness in complex service environments

Implementation is a crucial component for the success of interventions in health service systems, as failure to implement well can have detrimental impacts on the effectiveness of evidence-based practices. Therefore, evaluations conducted in real-world contexts should consider how interventions are implemented and sustained. However, the complexity of healthcare environments poses considerable challenges to the evaluation of interventions and the impact of implementation efforts on the effectiveness of evidence-based practices. In consequence, implementation and intervention effectiveness are often assessed separately in health services research, which prevents the direct investigation of the relationships of implementation components and effectiveness of the intervention. This article describes multilevel decision juncture models based on advances in implementation research and causal inference to study implementation in health service systems. The multilevel decision juncture model is a theory-driven systems approach that integrates structural causal models with frameworks for implementation. This integration enables investigation of interventions and their implementation within a single model that considers the causal links between levels of the system. Using a hypothetical youth mental health intervention inspired by published studies from the health service research and implementation literature, we demonstrate that such theory-based systems models enable investigations of the causal pathways between the implementation outcomes as well as their links to patient outcomes. Results from Monte Carlo simulations also highlight the benefits of structural causal models for covariate selection as consistent estimation requires only the inclusion of a minimal set of covariates. Such models are applicable to real-world context using different study designs, including longitudinal analyses which facilitates the investigation of sustainment of interventions.

Critical review and quality-assessment of cost analyses in radiotherapy: How reliable are the data?

PURPOSE/OBJECTIVE

Health economic evaluations (HEE) are increasingly having an impact on policymakers, although the results greatly depend on the quality of the methodology used and on transparent reporting. The two main objectives of this study were to evaluate the quality of cost analyses of external beam radiotherapy (EBRT) and to assess the comprehensiveness and relevance of cost criteria defined in three validated quality-assessment instruments.

MATERIALS AND METHODS

The selection of articles was based on a previous systematic literature review of EBRT-costing studies retrieved from January 2004 to January 2015 (Period 1) in MEDLINE, Embase, and NHS-EED databases and completed in a second time period from January 2015 to November 2018 (Period 2). Three validated instruments to assess the methodology quality with the CHEC and the QHES, and the methodology with the CHEERS checklists were used. The quality was evaluated by both quantitative and qualitative analyses. The scoring robustness was examined with the Kendall coefficient of concordance and inter-class correlation coefficients.

RESULTS

In total, twenty-three articles were selected. The main geographic areas of cost analyses were Canada (n = 5), France (n = 4), and the USA (n = 4). The most commonly studied pathologies and technologies were prostate (n = 7) and head and neck cancer (n = 5) and IMRT (n = 8) and IGRT (n = 2), respectively. The mean instrument scores demonstrated a fair degree of methodological quality, with 69.7% for the CHEC, 73.6% for the QHES, as well as for the reporting quality, with 59.4% for CHEERS for Period 1 (74.4%, 71.5%, and 66.1%, respectively, for Period 2). An additional qualitative analysis per criterion revealed that certain items, essential for understanding the costing methodology and the results (e.g., the time horizon, discount rate, sensitivity analysis) were often only partially completed. Statistical analysis confirmed that the reviewers' scoring was consistent. The instruments identified the same top three articles, albeit with a degree of variation in the ranking.

CONCLUSION

Qualitative and quantitative assessment of cost analyses in EBRT exhibits a fair level of study quality in terms of the methodology and reporting transparency. The impact of cost calculations on the final HEE result appears to be underestimated, and increased transparency of the data sources and the methodologies is needed.

How to critically appraise a qualitative health research study

Evidence-based nursing is a process that requires nurses to have the knowledge, skills, and confidence to critically reflect on their practice, articulate structured questions, and then reliably search for research evidence to address the questions posed. Many types of research evidence are used to inform decisions in health care and findings from qualita- tive health research studies are useful to provide new insights about individuals' experi- ences, values, beliefs, needs, or perceptions. Before qualitative evidence can be utilized in a decision, it must be critically appraised to determine if the findings are trustworthy and if they have relevance to the identified issue or decision. In this article, we provide practical guidance on how to select a checklist or tool to guide the critical appraisal of qualitative studies and then provide an example demonstrating how to apply the critical appraisal process to a clinical scenario.

Key stakeholders' perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol

INTRODUCTION

Identifying research gaps can inform the design and conduct of health research, practice and policies by informing the current body of evidence. Audiences including researchers, clinical guideline developers, clinicians, policymakers, research regulatory bodies, funders and patients/the public can also benefit from understanding the status of research and research gaps to make informed choices. This study aims to explore how key informants define research gaps and characterise methods/practices used to identify and display gaps in health research to inform future research practice and policies.

METHODS AND ANALYSIS

This is an exploratory qualitative study using semi-structured in-depth interviews. The participants will be recruited by purposive sampling from initiatives and organisations previously identified in a scoping review on methods to identify, prioritise and display gaps in health research. We anticipate performing up to 28 interviews with the different key informant groups who are involved in using evidence to inform health policy, practice and research. Interviews will be thematically analysed as outlined by Braun and Clarke. The qualitative data-analysis software NVivo V.12 Pro will be used to aid data management and analysis.

DISCUSSION

This is the protocol for a follow-up study that aims to complement and enrich the findings of the scoping review on methods to identify, prioritise and display gaps in health research. The overall project aims to develop methodological guidance for describing, identifying and displaying gaps in health research.

ETHICS AND DISSEMINATION

The research obtained ethical approval from the University of Liverpool, UK. The findings will be disseminated via conferences, meetings (organised by the Methods in Research on Research project), peer-reviewed publications and lay magazines because the study participants will include the public/patients.

Qualitative Evidence Synthesis (QES) for Guidelines: Paper 3 - Using qualitative evidence syntheses to develop implementation considerations and inform implementation processes

BACKGROUND

This is the third in a series of three papers describing the use of qualitative evidence syntheses (QES) to inform the development of clinical and health systems guidelines. WHO has recognised the need to improve its guideline methodology to ensure that decision-making processes are transparent and evidence based, and that the resulting recommendations are relevant and applicable to end users. In addition to the standard data on effectiveness, WHO guidelines increasingly use evidence derived from QES to provide information on acceptability and feasibility and to develop important implementation considerations.

METHODS

WHO convened a group drawn from the technical teams involved in formulating recent (2010-2018) guidelines employing QES. Using a pragmatic and iterative approach that included feedback from WHO staff and other stakeholders, the group reflected on, discussed and identified key methods and research implications from designing QES and using the resulting findings in guideline development. As members of WHO guideline technical teams, our aim in this paper is to explore how we have used findings from QES to develop implementation considerations for these guidelines.

RESULTS

For each guideline, in addition to using systematic reviews of effectiveness, the technical teams used QES to gather evidence of the acceptability and feasibility of interventions and, in some cases, equity issues and the value people place on different outcomes. This evidence was synthesised using standardised processes. The teams then used the QES to identify implementation considerations combined with other sources of information and input from experts.

CONCLUSIONS

QES were useful sources of information for implementation considerations. However, several issues for further development remain, including whether researchers should use existing health systems frameworks when developing implementation considerations; whether researchers should take confidence in the evidence into account when developing implementation considerations; whether qualitative evidence that reveals implementation challenges should lead guideline panels to make conditional recommendations or only point to implementation considerations; and whether guideline users find it helpful to have challenges pointed out to them or whether they also need solutions. Finally, we need to explore how QES findings can be incorporated into derivative products to aid implementation.

Qualitative Evidence Synthesis (QES) for Guidelines: Paper 2 - Using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations

BACKGROUND

WHO has recognised the need to improve its guideline methodology to ensure that guideline decision-making processes are transparent and evidence based, and that the resulting recommendations are relevant and applicable. To help achieve this, WHO guidelines now typically enhance intervention effectiveness data with evidence on a wider range of decision-making criteria, including how stakeholders value different outcomes, equity, gender and human rights impacts, and the acceptability and feasibility of interventions. Qualitative evidence syntheses (QES) are increasingly used to provide evidence on this wider range of issues. In this paper, we describe and discuss how to use the findings from QES to populate decision-making criteria in evidence-to-decision (EtD) frameworks. This is the second in a series of three papers that examines the use of QES in developing clinical and health system guidelines.

METHODS

WHO convened a writing group drawn from the technical teams involved in its recent (2010-2018) guidelines employing QES. Using a pragmatic and iterative approach that included feedback from WHO staff and other stakeholders, the group reflected on, discussed and identified key methods and research implications from designing QES and using the resulting findings in guideline development.

RESULTS

We describe a step-wise approach to populating EtD frameworks with QES findings. This involves allocating findings to the different EtD criteria (how stakeholders value different outcomes, equity, acceptability and feasibility, etc.), weaving the findings into a short narrative relevant to each criterion, and inserting this summary narrative into the corresponding 'research evidence' sections of the EtD. We also identify areas for further methodological research, including how best to summarise and present qualitative data to groups developing guidelines, how these groups draw on different types of evidence in their decisions, and the extent to which our experiences are relevant to decision-making processes in fields other than health.

CONCLUSIONS

This paper shows the value of incorporating QES within a guideline development process, and the roles that qualitative evidence can play in integrating the views and experiences of relevant stakeholders, including groups who may not be otherwise represented in the decision-making process.

Qualitative Evidence Synthesis (QES) for Guidelines: Paper 1 - Using qualitative evidence synthesis to inform guideline scope and develop qualitative findings statements

BACKGROUND

WHO has recognised the need to ensure that guideline processes are transparent and evidence based, and that the resulting recommendations are relevant and applicable. Along with decision-making criteria that require findings from effectiveness reviews, WHO is increasingly using evidence derived from qualitative evidence syntheses (QES) to inform the values, acceptability, equity and feasibility implications of its recommendations. This is the first in a series of three papers examining the use of QES in developing clinical and health systems guidelines.

METHODS

WHO convened a group of methodologists involved in developing recent (2010-2018) guidelines that were informed by QES. Using a pragmatic and iterative approach that included feedback from WHO staff and other stakeholders, the group reflected on, discussed and identified key methods and research implications from designing QES and using the resulting findings in guideline development. Our aim in this paper is to (1) describe and discuss how the findings of QES can inform the scope of a guideline and (2) develop findings for key guideline decision-making criteria.

RESULTS

QES resulted in the addition of new outcomes that are directly relevant to service users, a stronger evidence base for decisions about how much effective interventions and related outcomes are valued by stakeholders in a range of contexts, and a more complete database of summary evidence for guideline panels to consider, linked to decisions about values, acceptability, feasibility and equity.

CONCLUSIONS

Rigorously conducted QES can be a powerful means of improving the relevance of guidelines, and of ensuring that the concerns of stakeholders, at all levels of the healthcare system and from a wide range of settings, are taken into account at all stages of the process.

An Academic Research Coach: An Innovative Approach to Increasing Scholarly Productivity in Medicine

BACKGROUND

Academic faculty who devote most of their time to clinical work often struggle to engage in meaningful scholarly work. They may be disadvantaged by limited research training and limited time. Simply providing senior mentors and biostatistical support has limited effectiveness.

OBJECTIVE

We aimed to increase productivity in scholarly work of hospitalists and internal medicine physicians by integrating an Academic Research Coach into a robust faculty development program.

DESIGN

This was a pre-post quality improvement evaluation.

SETTING

This was conducted at the University of Washington in faculty across three academic-affiliated hospitals and 10 academic-affiliated clinics.

PARTICIPANTS

Participants were hospitalists and internists on faculty in the Division of General Internal Medicine at the University of Washington.

INTERVENTION

The coach was a 0.50 full time equivalent health services researcher with strong research methods, project implementation, and interpersonal skills. The coach consulted on research, quality improvement, and other scholarship.

MEASUREMENTS

We assessed the number of faculty supported, types of services provided, and numbers of grants, papers, and abstracts submitted and accepted.

RESULTS

The coach consulted with 49 general internal medicine faculty including 30 hospitalists who conducted 63 projects. The coach supported 13 publications, 11 abstracts, four grant submissions, and seven manuscript reviews. Forty-eight faculty in other departments benefited as co-authors.

CONCLUSION

Employing a dedicated health services researcher as part of a faculty development program is an effective way to engage clinically oriented faculty in meaningful scholarship. Key aspects of the program included an accessible and knowledgeable coach and an ongoing marketing strategy.