Problems with evidence assessment in COVID-19 health policy impact evaluation: a systematic review of study design and evidence strength

INTRODUCTION

Assessing the impact of COVID-19 policy is critical for informing future policies. However, there are concerns about the overall strength of COVID-19 impact evaluation studies given the circumstances for evaluation and concerns about the publication environment.

METHODS

We included studies that were primarily designed to estimate the quantitative impact of one or more implemented COVID-19 policies on direct SARS-CoV-2 and COVID-19 outcomes. After searching PubMed for peer-reviewed articles published on 26 November 2020 or earlier and screening, all studies were reviewed by three reviewers first independently and then to consensus. The review tool was based on previously developed and released review guidance for COVID-19 policy impact evaluation.

RESULTS

After 102 articles were identified as potentially meeting inclusion criteria, we identified 36 published articles that evaluated the quantitative impact of COVID-19 policies on direct COVID-19 outcomes. Nine studies were set aside because the study design was considered inappropriate for COVID-19 policy impact evaluation (n=8 pre/post; n=1 cross-sectional), and 27 articles were given a full consensus assessment. 20/27 met criteria for graphical display of data, 5/27 for functional form, 19/27 for timing between policy implementation and impact, and only 3/27 for concurrent changes to the outcomes. Only 4/27 were rated as overall appropriate. Including the 9 studies set aside, reviewers found that only four of the 36 identified published and peer-reviewed health policy impact evaluation studies passed a set of key design checks for identifying the causal impact of policies on COVID-19 outcomes.

DISCUSSION

The reviewed literature directly evaluating the impact of COVID-19 policies largely failed to meet key design criteria for inference of sufficient rigour to be actionable by policy-makers. More reliable evidence review is needed to both identify and produce policy-actionable evidence, alongside the recognition that actionable evidence is often unlikely to be feasible.

Curating and translating the evidence about SARS-CoV-2 and COVID-19 for frontline public health and clinical care: The Novel Coronavirus Research Compendium (NCRC)

The public health crisis created by the SARS-CoV-2 pandemic has spurred a deluge of scientific research aimed at informing public health and medical response to the COVID-19 pandemic. However, those working in frontline public health and clinical care had insufficient time to parse the rapidly evolving evidence and use it for decision making. Academics in public health and medicine were well-placed to translate the evidence for use by frontline clinicians and public health practitioners. The Novel Coronavirus Research Compendium (NCRC), a group of >50 faculty and trainees, began in March 2020 with the goal to quickly triage and review the large volume of preprints and peer-reviewed publications on SARS-CoV-2 and COVID-19, and to summarize the most important, novel evidence to inform pandemic response. From April 6, 2020 through January 1, 2021, 54,192 papers and preprints were screened by NCRC teams and 527 were selected for review and uploaded to the NCRC website for public consumption. The majority of papers reviewed were peer-reviewed publications (n=395, 75%), published in 102 journals; 25% (n=132) of papers reviewed were of preprints. The NCRC is a successful model of how academics can support practitioners by translating scientific knowledge into action and help to build capacity among students for this work. This approach could be used for health problems beyond COVID-19, but the effort is resource intensive and may not be sustainable over the long term.

Characterising the impact of COVID-19 environment on mental health, gender affirming services and socioeconomic loss in a global sample of transgender and non-binary people: a structural equation modelling

BACKGROUND

We characterised the impact of COVID-19 on the socioeconomic conditions, access to gender affirmation services and mental health outcomes in a sample of global transgender (trans) and non-binary populations.

METHODS

Between 16 April 2020 and 3 August 2020, we conducted a cross-sectional survey with a global sample of trans and non-binary people (n=849) through an online social networking app. We conducted structural equational modelling procedures to determine direct, indirect and overall effects between poor mental health (ie, depression and anxiety) and latent variables across socioecological levels: social (ie, reduction in gender affirming services, socioeconomic loss impact) and environmental factors (ie, COVID-19 pandemic environment).

RESULTS

Anxiety (45.82%) and depression (50.88%) in this sample were prevalent and directly linked to COVID-19 pandemic environment. Adjusted for gender identity, age, migrant status, region, education and level of socioeconomic status, our final model showed significant positive associations between relationships of (1) COVID-19 pandemic environment and socioeconomic loss impact (β=0.62, p<0.001), (2) socioeconomic loss impact and reduction in gender affirming services (β=0.24, p<0.05) and (3) reduction in gender affirming services and poor mental health (β=0.19, p<0.05). Moreover, socioeconomic loss impact and reduction in gender affirming services were found to be partial mediators in this model.

CONCLUSION

The study results supported the importance of bolstering access to gender affirming services and strengthening socioeconomic opportunities and programmatic support to buffer the impact of COVID-19 pandemic environment on poor mental health among trans and non-binary communities globally.

Formative research for an mHealth program to improve the HIV care continuum in South Africa

In South Africa, high attrition rates throughout the care continuum present major barriers to controlling the HIV epidemic. Mobile health (mHealth) interventions may provide innovative opportunities for efficient healthcare delivery and improving retention in care. In this formative research, we interviewed 11 patients and 28 healthcare providers in North West Province, South Africa, to identify perceived benefits, concerns and suggestions for a future mHealth program to deliver HIV Viral Load and CD4 Count test results directly to patients via mobile phone. Thematic analysis found that reduced workload for providers, reduced wait times for patients, potential expanded uses and patient empowerment were the main perceived benefits of an mHealth program. Perceived concerns included privacy, disseminating distressing results through text messages and patients' inability to interpret results. Participants felt that an mHealth program should complement face-to-face interactions and educational information to interpret results is needed. Providers identified logistical considerations and suggested protocols be developed. An mHealth program to deliver HIV test results directly to patients could mitigate multiple barriers to care but needs to be tested for efficacy. Concerns identified by patients and providers must be addressed in designing the program to successfully integrate with health facility workflow and ensure its sustainability.

Antimicrobial resistance of Neisseria gonorrhoeae in Liberia

The prevalence and molecular characteristics of penicillinase-producing Neisseria gonorrhoeae (PPNG) and tetracycline-resistant N. gonorrhoeae (TRNG) were determined in 10 clinics in Monrovia, Liberia, to assess the likely effectiveness of the current standard treatment with penicillin or tetracycline. One hundred gonococcal strains were isolated from 146 urethral swabs and 261 cervical swabs and screened for susceptibility to ceftriaxone, penicillin, spectinomycin and tetracycline by the disk diffusion method; 83% were resistant to penicillin and 63% to tetracycline. Twenty-one strains from 18 men and 3 women with uncomplicated gonorrhoea were subjected to more detailed characterization. These 21 strains belonged to 5 auxotype/serovar classes; 86% were PPNG/TRNG. Three PPNG harboured the 4.4 MDa penicillinase plasmid and 16 the 3.2 MDa plasmid. All TRNG harboured the 25.2 MDa plasmid and their MICs for tetracycline were > 32 mg/L. They gave a PCR product which, according to its restriction pattern, corresponded to the American type tetM gene. By the agar dilution method, all strains exhibited intermediate resistance to sulphamethoxazole-trimethoprim (19:1) (co-trimoxazole) with MICs of 8-32 mg/L. All strains were susceptible to spectinomycin and ciprofloxacin. The MICs for gentamicin were 4-8 mg/L. The use of effective and affordable antimicrobial chemotherapy with either 500 mg ciprofloxacin or a single dose of gentamicin is discussed, with consideration of molecular biological, pharmacological and public health aspects.

[Treatment of schistosomiasis haematobium with metritionate in OPD-patients (author's transl)]

40 patients--5 to 50 years of age--voiding eggs of Schistosoma haematobium in their urine--underwent treatment with Metrifonate at the OPD of Bong Mine Hospital, Liberia. The patients received 10 mg/kg body weight 3 times at a fortnight's interval. The drug was swallowed under medical supervision. 27 patients no longer passed eggs after the 1. dose of Metrifonate, 37 no longer voided eggs after the 2. administration. 1 patient did not show up for control after the 3. dose. Theoretically he may not be healed. 1 other patient who came for control after 12 months had been exposed to reinfection and again voided eggs in her urine. She had been negative after the 2. treatment. Reinfection may have happened. Patients could be controlled over a period of 6-14 months. Thus, Metrifonate seems to be an effective drug in the treatment of urinary schistosomiasis. Side-effects (nausea, abdominal pains, and--very rare--vomiting) were mild and disappeared spontaneously within less than 24 hours after medication. Patients did not have to interrupt school, daily activities, and treatment. Statistically, Metrifonate did not show any influence on transaminase SGOT, SGPT, and LDH during and after the course of treatment. The same evaluation applies to eosinophilia. There is no increase or decrease of this particular type of cell during and after treatment 7 patients showed alterations of their ECG curves. There were changes of the T-wave in V1-4. In adults traces were normal again several months after completion of treatment. It seems to be difficult to interpret ECGs in West African youngsters. X-ray photos of the lungs never revealed any pathological findings which could be connected to the course of treatment. Metrifonate seems to be a valuable drug in treatment of Schistosoma haematobium-infection. The drug is well tolerated if the treatment scheme--mentioned above--is used.