Economics methods in Cochrane systematic reviews of health promotion and public health related interventions

Primary-level and community worker interventions for the prevention of mental disorders and the promotion of well-being in low- and middle-income countries

BACKGROUND

There is a significant research gap in the field of universal, selective, and indicated prevention interventions for mental health promotion and the prevention of mental disorders. Barriers to closing the research gap include scarcity of skilled human resources, large inequities in resource distribution and utilization, and stigma.

OBJECTIVES

To assess the effectiveness of delivery by primary workers of interventions for the promotion of mental health and universal prevention, and for the selective and indicated prevention of mental disorders or symptoms of mental illness in low- and middle-income countries (LMICs). To examine the impact of intervention delivery by primary workers on resource use and costs.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, CINAHL, Global Index Medicus, PsycInfo, WHO ICTRP, and ClinicalTrials.gov from inception to 29 November 2021.

SELECTION CRITERIA

Randomized controlled trials (RCTs) of primary-level and/or community health worker interventions for promoting mental health and/or preventing mental disorders versus any control conditions in adults and children in LMICs.

DATA COLLECTION AND ANALYSIS

Standardized mean differences (SMD) or mean differences (MD) were used for continuous outcomes, and risk ratios (RR) for dichotomous data, using a random-effects model. We analyzed data at 0 to 1, 1 to 6, and 7 to 24 months post-intervention. For SMDs, 0.20 to 0.49 represented small, 0.50 to 0.79 moderate, and ≥ 0.80 large clinical effects. We evaluated the risk of bias (RoB) using Cochrane RoB2.

MAIN RESULTS

Description of studies We identified 113 studies with 32,992 participants (97 RCTs, 19,570 participants in meta-analyses) for inclusion. Nineteen RCTs were conducted in low-income countries, 27 in low-middle-income countries, 2 in middle-income countries, 58 in upper-middle-income countries and 7 in mixed settings. Eighty-three RCTs included adults and 30 RCTs included children. Cadres of primary-level workers employed primary care health workers (38 studies), community workers (71 studies), both (2 studies), and not reported (2 studies). Interventions were universal prevention/promotion in 22 studies, selective in 36, and indicated prevention in 55 RCTs. Risk of bias The most common concerns over risk of bias were performance bias, attrition bias, and reporting bias. Intervention effects 'Probably', 'may', or 'uncertain' indicates 'moderate-', 'low-', or 'very low-'certainty evidence. *Certainty of the evidence (using GRADE) was assessed at 0 to 1 month post-intervention as specified in the review protocol. In the abstract, we did not report results for outcomes for which evidence was missing or very uncertain. Adults Promotion/universal prevention, compared to usual care: - probably slightly reduced anxiety symptoms (MD -0.14, 95% confidence interval (CI) -0.27 to -0.01; 1 trial, 158 participants) - may slightly reduce distress/PTSD symptoms (SMD -0.24, 95% CI -0.41 to -0.08; 4 trials, 722 participants) Selective prevention, compared to usual care: - probably slightly reduced depressive symptoms (SMD -0.69, 95% CI -1.08 to -0.30; 4 trials, 223 participants) Indicated prevention, compared to usual care: - may reduce adverse events (1 trial, 547 participants) - probably slightly reduced functional impairment (SMD -0.12, 95% CI -0.39 to -0.15; 4 trials, 663 participants) Children Promotion/universal prevention, compared to usual care: - may improve the quality of life (SMD -0.25, 95% CI -0.39 to -0.11; 2 trials, 803 participants) - may reduce adverse events (1 trial, 694 participants) - may slightly reduce depressive symptoms (MD -3.04, 95% CI -6 to -0.08; 1 trial, 160 participants) - may slightly reduce anxiety symptoms (MD -2.27, 95% CI -3.13 to -1.41; 1 trial, 183 participants) Selective prevention, compared to usual care: - probably slightly reduced depressive symptoms (SMD 0, 95% CI -0.16 to -0.15; 2 trials, 638 participants) - may slightly reduce anxiety symptoms (MD 4.50, 95% CI -12.05 to 21.05; 1 trial, 28 participants) - probably slightly reduced distress/PTSD symptoms (MD -2.14, 95% CI -3.77 to -0.51; 1 trial, 159 participants) Indicated prevention, compared to usual care: - decreased slightly functional impairment (SMD -0.29, 95% CI -0.47 to -0.10; 2 trials, 448 participants) - decreased slightly depressive symptoms (SMD -0.18, 95% CI -0.32 to -0.04; 4 trials, 771 participants) - may slightly reduce distress/PTSD symptoms (SMD 0.24, 95% CI -1.28 to 1.76; 2 trials, 448 participants).

AUTHORS' CONCLUSIONS

The evidence indicated that prevention interventions delivered through primary workers - a form of task-shifting - may improve mental health outcomes. Certainty in the evidence was influenced by the risk of bias and by substantial levels of heterogeneity. A supportive network of infrastructure and research would enhance and reinforce this delivery modality across LMICs.

Community Guide Methods for Systematic Reviews of Economic Evidence

INTRODUCTION

Community Guide systematic economic reviews provide information on the cost, economic benefit, cost-benefit, and cost-effectiveness of public health interventions recommended by the Community Preventive Services Task Force on the basis of evidence of effectiveness. The number and variety of economic evaluation studies in public health have grown substantially over time, contributing to methodologic challenges that required updates to the methods for Community Guide systematic economic reviews. This paper describes these updated methods.

METHODS

The 9-step Community Guide economic review process includes prioritization of topic, creation of a coordination team, conceptualization of review, literature search, screening studies for inclusion, abstraction of studies, analysis of results, translation of evidence to Community Preventive Services Task Force economic findings, and dissemination of findings and evidence gaps. The methods applied in each of these steps are reported in this paper.

RESULTS

Two published Community Guide reviews, tailored pharmacy-based interventions to improve adherence to medications for cardiovascular disease and permanent supportive housing with housing first to prevent homelessness, are used to illustrate the application of the updated methods. The Community Preventive Services Task Force reached a finding of cost-effectiveness for the first intervention and a finding of favorable cost-benefit for the second on the basis of results from the economic reviews.

CONCLUSIONS

The updated Community Guide economic systematic review methods provide transparency and improve the reliability of estimates that are used to derive a Community Preventive Services Task Force economic finding. This may in turn augment the utility of Community Guide economic reviews for communities making decisions about allocating limited resources to effective programs.

The cost-effectiveness of tranexamic acid for treatment of postpartum hemorrhage: A systematic review

BACKGROUND

Postpartum hemorrhage (PPH) is responsible for nearly one quarter of maternal deaths. A 2017 multicountry trial found that incorporating tranexamic acid (TXA) into the PPH management package was effective in reducing maternal death due to bleeding.

OBJECTIVE

To systematically review studies assessing the cost-effectiveness of tranexamic acid for PPH treatment.

SEARCH STRATEGY

Nine databases were searched using variations of keywords 'tranexamic acid', 'postpartum hemorrhage' and 'cost effectiveness'.

SELECTION CRITERIA

Eligible studies were any type of economic or effectiveness evaluation studies on tranexamic acid for treating women with PPH.

DATA COLLECTION AND ANALYSIS

Two reviewers independently screened citations and extracted data on cost effectiveness measures. Quality was assessed using the Consensus on Health Economic Criteria list.

MAIN RESULTS

Four studies were included, of which two were abstracts. Three studies concluded that early administration of TXA was cost-saving or cost-effective. One abstract reported TXA was not cost-effective in the USA unless the probability of death due to hemorrhage is higher.

CONCLUSION

Available evidence (four studies in three countries) suggests that this life-saving intervention may be below willingness to pay thresholds (cost-effective) or cost saving. Further studies conducted in different populations and settings are needed to inform health policy decision-making to reduce PPH-associated morbidity and mortality.

Challenges of Systematic Reviews of Economic Evaluations: A Review of Recent Reviews and an Obesity Case Study

Decision makers increasingly require cost-effectiveness evidence to inform resource allocation and the need for systematic reviews of economic evaluations (SREEs) has grown accordingly. The objective of this article is to describe current practice and identify unique challenges in conducting and reporting SREEs. Current guideline documents for SREEs were consulted and summarised. A rapid review of English-language SREEs, using MEDLINE and EMBASE, published in 2017/2018, containing at least 20 studies was undertaken to describe current practice. Information on data extraction methods, quality assessment (QA) tools and reporting methods were narratively summarised. Lessons learned from a recently conducted SREE of weight loss interventions for severely obese adults were discussed. Sixty-three publications were included in the rapid review. Substantial heterogeneity in review methods, reporting standards and QA approaches was evident. Our recently conducted SREE on weight loss interventions identified scope to improve process efficiency, opportunity for more transparent and succinct reporting, and potential to improve consistency of QA. Practical solutions may include (1) using pre-piloted data extraction forms linked explicitly to results tables; (2) consistently reporting on key assumptions and sensitivity analyses that drive results; and (3) using checklists that include topic-specific items where relevant and allow reviewers to distinguish between reporting, justification and QA. The lack of a mutually agreed, standardised set of best practice guidelines has led to substantial heterogeneity in the conduct and reporting of SREEs. Future work is required to standardise the approach to conducting SREEs so that they can generate efficient, timely and relevant evidence to support decision-making. We suggest only data extracting information that will be reported, focusing discussion around the key drivers of cost-effectiveness, and improving consistency in QA by distinguishing between what is reported, justified by authors and deemed appropriate by the reviewer.

Cost and cost-effectiveness of point-of-care testing and treatment for sexually transmitted and genital infections in pregnancy in low-income and middle-income countries: a systematic review protocol

INTRODUCTION

The economic and health burden of sexually transmitted and genital infections (henceforth, STIs) in low-income and middle-income countries (LMICs) is substantial. Left untreated, STIs during pregnancy may result in several adverse pregnancy and birth outcomes. Timely diagnosis and treatment at point-of-care (POC) can potentially improve these outcomes. Despite the availability and promotion of POC diagnostics for STIs as a key component of antenatal care in LMICs, their widespread use has been limited, owing to the high economic costs faced by individuals and health systems. To date, there have been no systematic reviews which explore the cost or cost-effectiveness of POC testing and treatment of STIs in pregnancy in LMICs. The objective of this protocol is to outline the methods that will compare, synthesise and appraise the existing literature in this domain.

METHODS AND ANALYSIS

We will conduct literature searches in MEDLINE, Embase and Web of Science. To find additional literature, we will search Google Scholar and hand search reference lists of included papers. Two reviewers will independently search databases, screen titles, abstracts and full texts; when necessary a third reviewer will resolve disputes. Only cost and cost-effectiveness studies of POC testing and treatment of STIs, including syphilis, chlamydia, trichomonas, gonorrhoea and bacterial vaginosis, in pregnancy in LMICs will be included. Published checklists will be used to assess quality of reporting practices and methodological approaches. We will also assess risk of publication bias. Interstudy heterogeneity will be assessed and depending on variation between studies, a meta-analysis or narrative synthesis will be conducted.

ETHICS AND DISSEMINATION

Ethical approval is not required as the review will use published literature. The results will be published in a peer-reviewed open source journal and presented at an international conference.

PROSPERO REGISTRATION NUMBER

CRD42018109072.

A systematic review of the quality and scope of economic evaluations in child oral health research

BACKGROUND

Economic evaluations provide policy makers with information to facilitate efficient resource allocation. To date, the quality and scope of economic evaluations in the field of child oral health has not been evaluated. Furthermore, whilst the involvement of children in research has been actively encouraged in recent years, the success of this movement in dental health economics has not yet been explored. This review aimed to determine the quality and scope of published economic evaluations applied to children's oral health and to consider the extent of children's involvement.

METHODS

The following databases were searched: CINAHL, Cochrane Library, Econlit, EThOS, MEDLINE, NHS EED, OpenGrey, Scopus, Web of Science. Full economic evaluations, relating to any aspect of child oral health, published after 1997 were included and appraised against the Drummond checklist and the Consolidated Health Economic Evaluation Reporting Standards by a team of four calibrated reviewers. Data were also extracted regarding children's involvement and the outcome measures used.

RESULTS

Two thousand seven hundred fifteen studies were identified, of which 46 met the inclusion criteria. The majority (n = 38, 82%) were cost-effectiveness studies, with most focusing on the prevention or management of dental caries (n = 42, 91%). One study quantified outcomes in Quality Adjusted Life Years (QALYs), and one study utilised a child-reported outcome measure. The mean percentage of applicable Drummond checklist criteria met by the studies in this review was 48% (median = 50%, range = 0-100%) with key methodological weaknesses noted in relation to discounting of costs and outcomes. The mean percentage of applicable CHEERS criteria met by each study was 77% (median = 83%, range = 33-100%), with limited reporting of conflicts of interest. Children's engagement was largely overlooked.

CONCLUSIONS

There is a paucity of high-quality economic evaluations in the field of child oral health. This deficiency could be addressed through the endorsement of standardised economic evaluation guidelines by dental journals. The development of a child-centred utility measure for use in paediatric oral health would enable researchers to quantify outcomes in terms of quality adjusted life years (QALYs) whilst promoting child-centred research.

A systematic review of decision analytic modeling techniques for the economic evaluation of dental caries interventions

OBJECTIVES

Dental caries occur through a multifactorial process that may influence all tooth surfaces throughout an individual's life. The application of decision analytical modeling (DAM) has gained an increasing level of attention in long-term outcome assessment and economic evaluation of interventions on caries in recent years. The objective of this study was to systematically review the application of DAM and assess their methodological quality in the context of dental caries.

METHODS

A systematic review of the literature published to 31st December 2018 was conducted in Medline, EMBASE, NHSEED, and Web of Science electronic databases. The main information and model characteristics of studies was extracted with the methodological quality of included studies reviewed and assessed using the Philips' checklist.

RESULTS

Twenty five studies from different settings were included. Modeling techniques mainly comprised main type of modeling including Markov models (n = 12), Markov/microsimulation mixed model (n = 7), systematic dynamic models (n = 3), microsimulation models (n = 2) and decision tree (n = 1). The mean number of criteria fulfilled was 25.1 out of 60 items, which varied between year of study and research groups. The percentage of criteria fulfilled for data dimension was lower than for the structure and consistency dimension. Critical issues were data quality, incorporation of utility values, and uncertainty analysis in modeling.

CONCLUSION

The current review revealed that the methodological quality of DAM in dental caries economic evaluations is unsatisfied. Future modeling studies should adhere more closely to good practice guidelines, especially with respect to data quality evaluation, utility values incorporation, and uncertainty analysis in DAM based studies.

A Review of Clinical Trials With an Adaptive Design and Health Economic Analysis

OBJECTIVE

An adaptive design uses data collected as a clinical trial progresses to inform modifications to the trial. Hence, adaptive designs and health economics aim to facilitate efficient and accurate decision making. Nevertheless, it is unclear whether the methods are considered together in the design, analysis, and reporting of trials. This review aims to establish how health economic outcomes are used in the design, analysis, and reporting of adaptive designs.

METHODS

Registered and published trials up to August 2016 with an adaptive design and health economic analysis were identified. The use of health economics in the design, analysis, and reporting was assessed. Summary statistics are presented and recommendations formed based on the research team's experiences and a practical interpretation of the results.

RESULTS

Thirty-seven trials with an adaptive design and health economic analysis were identified. It was not clear whether the health economic analysis accounted for the adaptive design in 17/37 trials where this was thought necessary, nor whether health economic outcomes were used at the interim analysis for 18/19 of trials with results. The reporting of health economic results was suboptimal for the (17/19) trials with published results.

CONCLUSIONS

Appropriate consideration is rarely given to the health economic analysis of adaptive designs. Opportunities to use health economic outcomes in the design and analysis of adaptive trials are being missed. Further work is needed to establish whether adaptive designs and health economic analyses can be used together to increase the efficiency of health technology assessments without compromising accuracy.

How Much Do We Spend? Creating Historical Estimates of Public Health Expenditures in the United States at the Federal, State, and Local Levels

The United States has a complex governmental public health system. Agencies at the federal, state, and local levels all contribute to the protection and promotion of the population's health. Whether the modern public health system is well situated to deliver essential public health services, however, is an open question. In some part, its readiness relates to how agencies are funded and to what ends. A mix of Federalism, home rule, and happenstance has contributed to a siloed funding system in the United States, whereby health agencies are given particular dollars for particular tasks. Little discretionary funding remains. Furthermore, tracking how much is spent, by whom, and on what is notoriously challenging. This review both outlines the challenges associated with estimating public health spending and explains the known sources of funding that are used to estimate and demonstrate the value of public health spending.

Five indicators were developed to assess the quality of reviews on preventive interventions

OBJECTIVES

The objective of the study was to develop quality indicators for preventive effectiveness and to evaluate their use with Cochrane Reviews of primary preventive interventions.

STUDY DESIGN AND SETTING

Based on the quality of care framework, we searched the literature to develop a set of quality indicators. Two authors applied the quality indicators independently to a sample of Cochrane systematic reviews of primary prevention.

RESULTS

Five quality indicators were developed: sample size, directness of evidence, adherence, harm, and costs. We applied the quality indicators to a random sample of 84 of a total of 264 Cochrane reviews of primary preventive interventions. Only 70% reviews (n = 59) complied with the indicator sample size, whereas 61% (n = 51) complied with directness of the outcome, 48% (n = 40) with adherence, 76% (n = 64) with harm, and 46% (n = 39) with the indicator cost.

CONCLUSION

Applying the five quality indicators is feasible. The quality of evidence in reviews of primary prevention can be substantially improved. Trialists and review authors should provide more information especially on adherence, costs, and indirectness of the outcome. Methodological research is needed on how to incorporate cost information in systematic reviews and how to better deal with indirectness.

Diverse approaches to the health economic evaluation of bariatric surgery: a comprehensive systematic review

BACKGROUND

Health economic evaluations inform healthcare resource allocation decisions for treatment options for obesity including bariatric/metabolic surgery. As an important advance on existing systematic reviews, we aimed to capture, summarize and synthesize a diverse range of economic evaluations on bariatric surgery.

METHODS

Studies were identified by electronic screening of all major biomedical/economic databases. Studies included if they reported any quantified health economic cost and/or consequence with a measure of effect for any type of bariatric surgery from 1995 to September 2015. Study screening, data extraction and synthesis followed international guidelines for systematic reviews.

RESULTS

Six thousand one hundred eighty-seven studies were initially identified. After two levels of screening, 77 studies representing 17 countries (56% USA) were included. Despite study heterogeneity, common themes emerged, and important gaps were identified. Most studies adopted the healthcare system/third-party payer perspective; reported costs were generally healthcare resource use (inpatient/shorter-term outpatient). Out-of-pocket costs to individuals, family members (travel time, caregiving) and indirect costs due to lost productivity were largely ignored. Costs due to reoperations/complications were not included in one-third of studies. Body-contouring surgery included in only 14%. One study evaluated long-term waitlisted patients. Surgery was cost-effective/cost-saving for severely obese with type 2 diabetes mellitus. Study quality was inconsistent.

DISCUSSION

There is a need for studies that assume a broader societal perspective (including out-of-pocket costs, costs to family and productivity losses) and longer-term costs (capture reoperations/complications, waiting, body contouring), and consequences (health-related quality-of-life). Full economic evaluation underpinned by reporting standards should inform prioritization of patients (e.g. type 2 diabetes mellitus with body mass index 30 to 34.9 kg/m(2) or long-term waitlisted) for surgery. © 2016 World Obesity.

Methods for Systematic Reviews of Health Economic Evaluations

Introduction. The quality of systematic reviews of health economic evaluations (SR-HE) is often limited because of methodological shortcomings. One reason for this poor quality is that there are no established standards for the preparation of SR-HE. The objective of this study is to compare existing methods and suggest best practices for the preparation of SR-HE. Methods. To identify the relevant methodological literature on SR-HE, a systematic literature search was performed in Embase, Medline, the National Health System Economic Evaluation Database, the Health Technology Assessment Database, and the Cochrane methodology register, and webpages of international health technology assessment agencies were searched. The study selection was performed independently by 2 reviewers. Data were extracted by one reviewer and verified by a second reviewer. On the basis of the overlaps in the recommendations for the methods of SR-HE in the included papers, suggestions for best practices for the preparation of SR-HE were developed. Results. Nineteen relevant publications were identified. The recommendations within them often differed. However, for most process steps there was some overlap between recommendations for the methods of preparation. The overlaps were taken as basis on which to develop suggestions for the following process steps of preparation: defining the research question, developing eligibility criteria, conducting a literature search, selecting studies, assessing the methodological study quality, assessing transferability, and synthesizing data. Discussion. The differences in the proposed recommendations are not always explainable by the focus on certain evaluation types, target audiences, or integration in the decision process. Currently, there seem to be no standard methods for the preparation of SR-HE. The suggestions presented here can contribute to the harmonization of methods for the preparation of SR-HE.

Issues in the incorporation of economic perspectives and evidence into Cochrane reviews

BACKGROUND

Methods for systematic reviews of the effects of health interventions have focused mainly on addressing the question of 'What works?' or 'Is this intervention effective in achieving one or more specific outcomes?' Addressing the question 'Is it worth it given the resources available?' has received less attention. This latter question can be addressed by applying an economic lens to the systematic review process.This paper reflects on the value and desire for the consideration by end users for coverage of an economic perspective in a Cochrane review and outlines two potential approaches and future directions.

METHODS

Two frameworks to guide review authors who are seeking to include an economic perspective are outlined. The first involves conducting a full systematic review of economic evaluations that is integrated into a review of intervention effects. The second involves developing a brief economic commentary. The two approaches share a set of common stages but allow the tailoring of the economic component of the Cochrane review to the skills and resources available to the review team.

RESULTS

The number of studies using the methods outlined in the paper is limited, and further examples are needed both to explore the value of these approaches and to further develop them. The rate of progress will hinge on the organisational leadership, capacity and resources available to the CCEMG, author teams and other Cochrane entities. Particular methodological challenges to overcome relate to understanding the key economic trade-offs and casual relationships for a given decision problem and informing the development of evaluations designed to support local decision-makers.

CONCLUSIONS

Methods for incorporating economic perspectives and evidence into Cochrane intervention reviews are established. Their role is not to provide a precise estimate of 'cost-effectiveness' but rather to help end-users of Cochrane reviews to determine the implications of the economic components of reviews for their own specific decisions.

Outcome measurement in economic evaluations of public health interventions: a role for the capability approach?

Public health interventions have received increased attention from policy makers, and there has been a corresponding increase in the number of economic evaluations within the domain of public health. However, methods to evaluate public health interventions are less well established than those for medical interventions. Focusing on health as an outcome measure is likely to underestimate the impact of many public health interventions. This paper provides a review of outcome measures in public health; and describes the benefits of using the capability approach as a means to developing an all encompassing outcome measure.