How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making?

'Leading from the front' implementation increases the success of influenza vaccination drives among healthcare workers: a reanalysis of systematic review evidence using Intervention Component Analysis (ICA) and Qualitative Comparative Analysis (QCA)

BACKGROUND

Seasonal influenza vaccination of healthcare workers (HCW) is widely recommended to protect staff and patients. A previous systematic review examined interventions to encourage uptake finding that hard mandates, such as loss of employment for non-vaccination, were more effective than soft mandates, such as signing a declination form, or other interventions such as incentives. Despite these overarching patterns the authors of the review concluded that 'substantial heterogeneity' remained requiring further analysis. This paper reanalyses the evidence using Intervention Component Analysis (ICA) and Qualitative Comparative Analysis (QCA) to examine whether the strategies used to implement interventions explain the residual heterogeneity.

METHODS

We used ICA to extract implementation features and trialists' reflections on what underpinned the success of the intervention they evaluated. The ICA findings then informed and structured two QCA analyses to systematically examine associations between implementation features and intervention outcomes. Analysis 1 examined hard mandate studies. Analysis 2 examined soft mandates and other interventions.

RESULTS

In Analysis 1 ICA revealed the significance of 'leading from the front' rather than 'top-down' implementation of hard mandates. Four key features underpinned this: providing education prior to implementation; two-way engagement so HCW can voice concerns prior to implementation; previous use of other strategies so that institutions 'don't-go-in-cold' with hard-mandates; and support from institutional leadership. QCA revealed that either of two configurations were associated with greater success of hard mandates. The first involves two-way engagement, leadership support and a 'don't-go-in-cold' approach. The second involves leadership support, education and a 'don't-go-in-cold' approach. Reapplying the 'leading from the front' theory in Analysis 2 revealed similar patterns.

CONCLUSIONS

Regardless of intervention type a 'leading from the front' approach to implementation will likely enhance intervention success. While the results pertain to flu vaccination among HCWs, the components identified here may be relevant to public health campaigns regarding COVID-19 vaccination.

Dynamic meta-analysis: a method of using global evidence for local decision making

Background

Meta-analysis is often used to make generalisations across all available evidence at the global scale. But how can these global generalisations be used for evidence-based decision making at the local scale, if the global evidence is not perceived to be relevant to local decisions? We show how an interactive method of meta-analysis—dynamic meta-analysis—can be used to assess the local relevance of global evidence.

Results

We developed Metadataset (www.metadataset.com) as a proof-of-concept for dynamic meta-analysis. Using Metadataset, we show how evidence can be filtered and weighted, and results can be recalculated, using dynamic methods of subgroup analysis, meta-regression, and recalibration. With an example from agroecology, we show how dynamic meta-analysis could lead to different conclusions for different subsets of the global evidence. Dynamic meta-analysis could also lead to a rebalancing of power and responsibility in evidence synthesis, since evidence users would be able to make decisions that are typically made by systematic reviewers—decisions about which studies to include (e.g. critical appraisal) and how to handle missing or poorly reported data (e.g. sensitivity analysis).

Conclusions

In this study, we show how dynamic meta-analysis can meet an important challenge in evidence-based decision making—the challenge of using global evidence for local decisions. We suggest that dynamic meta-analysis can be used for subject-wide evidence synthesis in several scientific disciplines, including agroecology and conservation biology. Future studies should develop standardised classification systems for the metadata that are used to filter and weight the evidence. Future studies should also develop standardised software packages, so that researchers can efficiently publish dynamic versions of their meta-analyses and keep them up-to-date as living systematic reviews. Metadataset is a proof-of-concept for this type of software, and it is open source. Future studies should improve the user experience, scale the software architecture, agree on standards for data and metadata storage and processing, and develop protocols for responsible evidence use.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-021-00974-w.

Clarifying differences between reviews within evidence ecosystems

This paper builds on a 2012 paper by the same authors which argued that the types and brands of systematic review do not sufficiently differentiate between the many dimensions of different review questions and review methods (Gough et al., Syst Rev 1:28, 2012). The current paper extends this argument by considering the dynamic contexts, or 'evidence ecosystems', within which reviews are undertaken; the fact that these ecosystems are constantly changing; and the relevance of this broader context for understanding 'dimensions of difference' in the unfolding development and refinement of review methods.The concept of an evidence ecosystem is used to consider particular issues within the three key dimensions of difference outlined in the 2012 paper of (1) review aims and approach, (2) structure and components of reviews, and (3) breadth, depth, and 'work done' by reviews.

Obstacles and opportunities to using research evidence in local public health decision-making in England

BACKGROUND

Local public health service delivery and policy-setting in England was overhauled in 2013, with local government now responsible for the complex tasks involved in protecting and improving population health and addressing health inequalities. Since 2013, public health funding per person has declined, adding to the challenge of public health decision-making. In a climate of austerity, research evidence could help to guide the more effective use of resources, although there are concerns that the reorganisation of public health decision-making structures has disrupted traditional evidence use patterns. This study aimed to explore local public health evidence use and needs in this new decision-making climate.

METHODS

Semi-structured interviews with Public Health Practitioners across three Local Authorities were conducted, with sites purposefully selected to represent urban, suburban and county Local Authorities, and to reflect a range of public health issues that might be encountered. A topic guide was developed that allowed participants to reflect on their experience and involvement in providing evidence for, or making a decision around, commissioning a public health service. Data were transcribed and template analysis was employed to understand the findings, which involved developing a coding template based on an initial transcript and applying this to subsequent transcripts.

RESULTS

Increased political involvement in local public health decision-making, while welcomed by some participants as a form of democratising public health, has influenced evidence preferences in a number of ways. Political and individual ideologies of locally elected officials meant that certain forms of evidence could be overlooked in favour of evidence that corresponded to decision-makers' preferences. Political involvement at the local level has increased the appetite for local knowledge and evidence. Research evidence needs to demonstrate its local salience if it is to contribute to decision-making alongside competing sources, particularly anecdotal information.

CONCLUSION

To better meet decision-making needs of politicians and practitioners, a shift in the scope of public health evidence is required. At a systematic review level, this could involve moving away from producing evidence that reflects broad global generalisations about narrow and simple questions, and instead towards producing forms of evidence that have local applicability and can support complex policy-focussed decisions.