Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm

Developing an Instrument to Evaluate the Quality of Dementia Websites

In today's digital era, health information, especially for conditions like dementia, is crucial. This study aims to develop an instrument, demenTia wEbsite measSurement insTrument (TEST), through four steps: identifying existing instruments, determining criteria, selecting and revising measurement statements, and validating the instrument from March to August 2020. Five health informatics experts used the content validity ratio (CVR) test for validation. Thirteen evaluators compared Fleiss Kappa and intraclass correlation coefficient (ICC) values across four dementia websites using TEST and another tool, DISCERN. TEST consists of seven criteria and 25 measurement statements focusing on content quality (relevance, credibility, currency) and user experience (accessibility, interactivity, attractiveness, privacy). CVR = 1 confirmed all statements as essential. The TEST demonstrated stronger consistency and assessor agreement compared to DISCERN, measured by Fleiss Kappa and ICC. Overall, it is a robust tool for reliable and user-friendly dementia resources, ensuring health holistic information accessibility.

[GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence]

OBJECTIVES

The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process.

STUDY DESIGN AND SETTING

Consensus-based guidance developed by the GRADE working group members and other methodologists. This is a German translation of the original paper published in English.

RESULTS

We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings.

CONCLUSION

The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society.

The human risks of bias in medical and rehabilitation research and practice: the eight Is

BACKGROUND

The human mind is liable to make biased interpretations. The biomedical paradigm supposes that an objective truth can be reached with rigorous scientific methods. Human risk of bias is defined in this paper as threats to validity of study results that cannot be controlled even by rigorous scientific methods.

AIM

To identify categories of potential human risks of biases in quantitative medical and rehabilitation practice and research; and to review the available evidence of human risks of biases in each of these categories.

DESIGN

A narrative review.

RESULTS

Eight potential categories of human risk of bias (HRoB) in clinical and rehabilitation practice and research were identified: identity, integrity, independence, intelligence, ideology, interest, incentive and inequity. There is evidence that conflict of interest, deficient integrity and inequity contribute to biased scientific interpretations; and that due to inequity the evidence of effectiveness of interventions for disadvantaged patient groups is scarce. There is also evidence that biased decisions can occur in health care practice due to the state of inequity.

DISCUSSION

As all categories of HRoB's start with the letter 'I', these are named the eight Is. The categories overlap with each other. Studies assessing the impacts of biases related to conflict of interests, deficient integrity and inequity were identified. However, other potential human risks of biases have been studied very little or not at all.

CONCLUSIONS

The human beings' liability to biased thinking and all categories of HRoB should be recognized in clinical and rehabilitation practice and research, and use the best verified means to reduce the HRoB. More research is needed, particularly on how to reduce HRoB in medical practice and quantitative clinical research. These actions should be considered congruent to those aiming to increase the validity of the scientific method.

CLINICAL REHABILITATION IMPACT

The human risk of bias should be considered in assessment of evidence from clinical epidemiology as well as in clinical praxis.

Consideration of sex and gender in Cochrane reviews of interventions for preventing healthcare-associated infections: a methodology study

BACKGROUND

Healthcare-associated infections (HAIs) are common and increase morbidity, mortality, and healthcare costs. Their control continues to be an unresolved issue worldwide. HAIs epidemiology shows sex/gender differences. Thus the lack of consideration of sex/gender in Cochrane reviews will limit their applicability and capacity to support informed decisions. This study aims to describe the extent to which Cochrane reviews of interventions for preventing HAIs consider sex and gender.

METHODS

Methodology study appraising Cochrane reviews of interventions to prevent HAIs.

SEARCH METHODS

Cochrane Database of Systematic Reviews from 1995 (launch of the journal) to 31 December 2016. Two authors independently extracted data with EPPI-Reviewer 4 software, and independently appraised the sex/gender content of the reviews with the Sex and Gender Appraisal Tool for Systematic Reviews (SGAT-SR).

RESULTS

This study included 113 reviews assessing the effects of interventions for preventing HAIs. 100 reviews (88%) used at least one sex or gender-related term. The terminology used was heterogeneous, being "sex" the term used in more reviews (51%). No review defined neither sex nor gender. Thus we could not assess the definitions provided. Consideration of sex and gender was practically absent in the included reviews; in fact, no review met all the applicable items of the SGAT-SR, and 51 reviews (50%) fulfilled no item. No review provided a complete description of the sex and the gender of the samples of the included studies. Only ten reviews (10%) planned to perform sex- and gender-based analysis and only three (3%) could complete the analysis. The method chosen was always the subgroup analysis based on sex (one review) or gender (two reviews). Three reviews (3%) considered sex or gender-related findings in the conclusions.

CONCLUSION

Consideration of sex and gender in Cochrane reviews of interventions for preventing HAIs was practically absent. This lack of attention to sex and gender reduces the quality of Cochrane reviews, and their applicability for all people: women and men, boys and girls, and people of diverse gender identities. Cochrane should attempt to address the shortfalls detected.

Best practices in nutrition science to earn and keep the public's trust

Public trust in nutrition science is the foundation on which nutrition and health progress is based, including sound public health. An ASN-commissioned, independent Advisory Committee comprehensively reviewed the literature and available public surveys about the public's trust in nutrition science and the factors that influence it and conducted stakeholder outreach regarding publicly available information. The Committee selected 7 overlapping domains projected to significantly influence public trust: 1) conflict of interest and objectivity; 2) public benefit; 3) standards of scientific rigor and reproducibility; 4) transparency; 5) equity; 6) information dissemination (education, communication, and marketing); and 7) accountability. The literature review comprehensively explored current practices and threats to public trust in nutrition science, including gaps that erode trust. Unfortunately, there is a paucity of peer-reviewed material specifically focused on nutrition science. Available material was examined, and its analysis informed the development of priority best practices. The Committee proposed best practices to support public trust, appropriate to ASN and other food and nutrition organizations motivated by the conviction that public trust remains key to the realization of the benefits of past, present, and future scientific advances. The adoption of the best practices by food and nutrition organizations, such as ASN, other stakeholder organizations, researchers, food and nutrition professionals, companies, government officials, and individuals working in the food and nutrition space would strengthen and help ensure earning and keeping the public's continued trust in nutrition science.

Toward a theory-led metaframework for considering socioeconomic health inequalities within systematic reviews

OBJECTIVES

To develop a theory-led framework to inform reviewers' understanding of what, how, and why health care interventions may lead to differential effects across socioeconomic groups.

STUDY DESIGN AND SETTING

A metaframework approach combined two theoretical perspectives (socioeconomic health inequalities and complex interventions) into a single framework to inform socioeconomic health inequality considerations in systematic reviews.

RESULTS

Four theories relating to complexity within systematic reviews and 16 health inequalities intervention theories informed the development of a metaframework. Factors relating to the type of intervention, implementation, context, participant response, and mechanisms associated with differential effects across socioeconomic groups were identified. The metaframework can inform; reviewer discussions around how socioeconomic status (SES) can moderate intervention effectiveness during question formulation, approaches to data extraction and help identify a priori analysis considerations.

CONCLUSION

The metaframework offers a transparent, practical, theory-led approach to inform a program theory for what, how, and why interventions work for different SES groups in systematic reviews. It can enhance existing guidance on conducting systematic reviews that consider health inequalities, increase awareness of how SES can moderate intervention effectiveness, and encourage a greater engagement with theory throughout the review process.

Use of programme theory to understand the differential effects of interventions across socio-economic groups in systematic reviews-a systematic methodology review

BACKGROUND

Systematic review guidance recommends the use of programme theory to inform considerations of if and how healthcare interventions may work differently across socio-economic status (SES) groups. This study aimed to address the lack of detail on how reviewers operationalise this in practice.

METHODS

A methodological systematic review was undertaken to assess if, how and the extent to which systematic reviewers operationalise the guidance on the use of programme theory in considerations of socio-economic inequalities in health. Multiple databases were searched from January 2013 to May 2016. Studies were included if they were systematic reviews assessing the effectiveness of an intervention and included data on SES. Two reviewers independently screened all studies, undertook quality assessment and extracted data. A narrative approach to synthesis was adopted.

RESULTS

A total of 37 systematic reviews were included, 10 of which were explicit in the use of terminology for 'programme theory'. Twenty-nine studies used programme theory to inform both their a priori assumptions and explain their review findings. Of these, 22 incorporated considerations of both what and how interventions do/do not work in SES groups to both predict and explain their review findings. Thirteen studies acknowledged 24 unique theoretical references to support their assumptions of what or how interventions may have different effects in SES groups. Most reviewers used supplementary evidence to support their considerations of differential effectiveness. The majority of authors outlined a programme theory in the "Introduction" and "Discussion" sections of the review to inform their assumptions or provide explanations of what or how interventions may result in differential effects within or across SES groups. About a third of reviews used programme theory to inform the review analysis and/or synthesis. Few authors used programme theory to inform their inclusion criteria, data extraction or quality assessment. Twenty-one studies tested their a priori programme theory.

CONCLUSIONS

The use of programme theory to inform considerations of if, what and how interventions lead to differential effects on health in different SES groups in the systematic review process is not yet widely adopted, is used implicitly, is often fragmented and is not implemented in a systematic way.

HEALTH TECHNOLOGY ASSESSMENT OF PUBLIC HEALTH INTERVENTIONS: A SYNTHESIS OF METHODOLOGICAL GUIDANCE

Objectives: The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI.

Methods: We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way.

Results: Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague.

Conclusions: The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.

GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence

Objectives

The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process.

Study Design and Setting

Consensus-based guidance developed by the GRADE working group members and other methodologists.

Results

We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings.

Conclusion

The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society.

Consideration of health inequalities in systematic reviews: a mapping review of guidance

BACKGROUND

Given that we know that interventions shown to be effective in improving the health of a population may actually widen the health inequalities gap while others reduce it, it is imperative that all systematic reviewers consider how the findings of their reviews may impact (reduce or increase) on the health inequality gap. This study reviewed existing guidance on incorporating considerations of health inequalities in systematic reviews in order to examine the extent to which they can help reviewers to incorporate such issues.

METHODS

A mapping review was undertaken to identify guidance documents that purported to inform reviewers on whether and how to incorporate considerations of health inequalities. Searches were undertaken in Medline, CINAHL and The Cochrane Library Methodology Register. Review guidance manuals prepared by international organisations engaged in undertaking systematic reviews, and their associated websites were scanned. Studies were included if they provided an overview or discussed the development and testing of guidance for dealing with the incorporation of considerations of health inequalities in evidence synthesis. Results are summarised in narrative and tabular forms.

RESULTS

Twenty guidance documents published between 2009 and 2016 were included. Guidance has been produced to inform considerations of health inequalities at different stages of the systematic review process. The Campbell and Cochrane Equity Group have been instrumental in developing and promoting such guidance. Definitions of health inequalities and guidance differed across the included studies. All but one guidance document were transparent in their method of production. Formal methods of evaluation were reported for six guidance documents. Most of the guidance was operationalised in the form of examples taken from published systematic reviews. The number of guidance items to operationalise ranges from 3 up to 26 with a considerable overlap noted.

CONCLUSIONS

Adhering to the guidance will require more work for the reviewers. It requires a deeper understanding of how reviewers can operationalise the guidance taking into consideration the barriers and facilitators involved. This has implications not only for understanding the usefulness and burden of the guidance but also for the uptake of guidance and its ultimate goal of improving health inequalities considerations in systematic reviews.

Interactive social media interventions to promote health equity: an overview of reviews

INTRODUCTION

Social media use has been increasing in public health and health promotion because it can remove geographic and physical access barriers. However, these interventions also have the potential to increase health inequities for people who do not have access to or do not use social media. In this paper, we aim to assess the effects of interactive social media interventions on health outcomes, behaviour change and health equity.

METHODS

We conducted a rapid response overview of systematic reviews. We used a sensitive search strategy to identify systematic reviews and included those that focussed on interventions allowing two-way interaction such as discussion forums, social networks (e.g. Facebook and Twitter), blogging, applications linked to online communities and media sharing.

RESULTS

Eleven systematic reviews met our inclusion criteria. Most interventions addressed by the reviews included online discussion boards or similar strategies, either as stand-alone interventions or in combination with other interventions. Seven reviews reported mixed effects on health outcomes and healthy behaviours. We did not find disaggregated analyses across characteristics associated with disadvantage, such as lower socioeconomic status or age. However, some targeted studies reported that social media interventions were effective in specific populations in terms of age, socioeconomic status, ethnicities and place of residence. Four reviews reported qualitative benefits such as satisfaction, finding information and improved social support.

CONCLUSION

Social media interventions were effective in certain populations at risk for disadvantage (youth, older adults, low socioeconomic status, rural), which indicates that these interventions may be effective for promoting health equity. However, confirmation of effectiveness would require further study. Several reviews raised the issue of acceptability of social media interventions. Only four studies reported on the level of intervention use and all of these reported low use. More research on established social media platforms with existing social networks is needed, particularly in populations at risk for disadvantage, to assess effects on health outcomes and health equity.

Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration

BACKGROUND

The promotion of health equity, the absence of avoidable and unfair differences in health outcomes, is a global imperative. Systematic reviews are an important source of evidence for health decision-makers, but have been found to lack assessments of the intervention effects on health equity. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) is a 27 item checklist intended to improve transparency and reporting of systematic reviews. We developed an equity extension for PRISMA (PRISMA-E 2012) to help systematic reviewers identify, extract, and synthesise evidence on equity in systematic reviews.

METHODS AND FINDINGS

In this explanation and elaboration paper we provide the rationale for each extension item. These items are additions or modifications to the existing PRISMA Statement items, in order to incorporate a focus on equity. An example of good reporting is provided for each item as well as the original PRISMA item.

CONCLUSIONS

This explanation and elaboration document is intended to accompany the PRISMA-E 2012 Statement and the PRISMA Statement to improve understanding of the reporting guideline for users. The PRISMA-E 2012 reporting guideline is intended to improve transparency and completeness of reporting of equity-focused systematic reviews. Improved reporting can lead to better judgement of applicability by policy makers which may result in more appropriate policies and programs and may contribute to reductions in health inequities. To encourage wide dissemination of this article it is accessible on the International Journal for Equity in Health, Journal of Clinical Epidemiology, and Journal of Development Effectiveness web sites.

On decreasing inequality in health care in a cost-effective way

Background

Scientific evidence indicates that access to health care services shown effective is often poorer among the disadvantaged people than among the non-disadvantaged. Advancing cost-effectiveness and equality in obtaining services shown effective are often stated as the two main objectives for health care.

Discussion

The important question is how to decrease inequality in a most cost-effective way. Some evidence indicates that effectiveness and cost-effectiveness of interventions may be high among the disadvantaged patient groups. However, the relationship between decreasing inequality and cost-effectiveness is complex and depend e.g. on clinical and health care system factors.

Summary

To expand the knowledge base for advancing equality in a cost-effective way, scientific efforts on four levels are suggested: research into ways how practitioners can provide efficient services for the disadvantaged patients, widening of scope at all levels of evidence based medicine to account also for the disadvantaged, research on quality improvement and evidence implementation, as well as benchmarking research including documentation of services among the disadvantaged.