Simulation-based power and sample size calculation for designing interrupted time series analyses of count outcomes in evaluation of health policy interventions

Cholecystectomy and digestive cancer in Chile: Complementary results from interrupted time series and aggregated data analyses

Gallbladder cancer (GBC) mortality in Chile is among the highest worldwide. In 2006, the Chilean government launched a programme guaranteeing access to gallbladder surgery (cholecystectomy) for patients aged 35-49 years. We evaluated the impact of this programme on digestive cancer mortality. After conducting an interrupted time series analysis of hospitalisation and mortality data from 2002 to 2018 publicly available from the Chilean Department of Health Statistics and Information, we calculated the change in the proportion of individuals without gallbladder since 10 years. We then estimated age, gender, region, and calendar-year standardised mortality ratios (SMRs) as a function of the change in the proportion of individuals without gallbladder. The cholecystectomy rate increased by 45 operations per 100,000 persons per year (95%CI 19-72) after the introduction of the health programme. Each 1% increase in the proportion of individuals without gallbladder since 10 years was associated with a 0.73% decrease in GBC mortality (95% CI -1.05% to -0.38%), but the negative correlation was limited to women, southern Chile and age over 60. We also found decreasing mortality rates for extrahepatic bile duct, liver, oesophageal and stomach cancer with increasing proportions of individuals without gallbladder. To conclude, 12 years after its inception, the Chilean cholecystectomy programme has markedly and heterogeneously changed cholecystectomy rates. Results based on aggregate data indicate a negative correlation between the proportion of individuals without gallbladder and mortality due to gallbladder and other digestive cancers, which requires validation using individual-level longitudinal data to reduce the potential impact of ecological bias.

Design and statistical analysis reporting among interrupted time series studies in drug utilization research: a cross-sectional survey

INTRODUCTION

Interrupted time series (ITS) design is a commonly used method for evaluating large-scale interventions in clinical practice or public health. However, improperly using this method can lead to biased results.

OBJECTIVE

To investigate design and statistical analysis characteristics of drug utilization studies using ITS design, and give recommendations for improvements.

METHODS

A literature search was conducted based on PubMed from January 2021 to December 2021. We included original articles that used ITS design to investigate drug utilization without restriction on study population or outcome types. A structured, pilot-tested questionnaire was developed to extract information regarding study characteristics and details about design and statistical analysis.

RESULTS

We included 153 eligible studies. Among those, 28.1% (43/153) clearly explained the rationale for using the ITS design and 13.7% (21/153) clarified the rationale of using the specified ITS model structure. One hundred and forty-nine studies used aggregated data to do ITS analysis, and 20.8% (31/149) clarified the rationale for the number of time points. The consideration of autocorrelation, non-stationary and seasonality was often lacking among those studies, and only 14 studies mentioned all of three methodological issues. Missing data was mentioned in 31 studies. Only 39.22% (60/153) reported the regression models, while 15 studies gave the incorrect interpretation of level change due to time parameterization. Time-varying participant characteristics were considered in 24 studies. In 97 studies containing hierarchical data, 23 studies clarified the heterogeneity among clusters and used statistical methods to address this issue.

CONCLUSION

The quality of design and statistical analyses in ITS studies for drug utilization remains unsatisfactory. Three emerging methodological issues warranted particular attention, including incorrect interpretation of level change due to time parameterization, time-varying participant characteristics and hierarchical data analysis. We offered specific recommendations about the design, analysis and reporting of the ITS study.

Kentucky Outreach Service Kiosk (KyOSK) Study protocol: a community-level, controlled quasi-experimental, type 1 hybrid effectiveness study to assess implementation, effectiveness and cost-effectiveness of a community-tailored harm reduction kiosk on HIV, HCV and overdose risk in rural Appalachia

INTRODUCTION

Many rural communities bear a disproportionate share of drug-related harms. Innovative harm reduction service models, such as vending machines or kiosks, can expand access to services that reduce drug-related harms. However, few kiosks operate in the USA, and their implementation, impact and cost-effectiveness have not been adequately evaluated in rural settings. This paper describes the Kentucky Outreach Service Kiosk (KyOSK) Study protocol to test the effectiveness, implementation outcomes and cost-effectiveness of a community-tailored, harm reduction kiosk in reducing HIV, hepatitis C and overdose risk in rural Appalachia.

METHODS AND ANALYSIS

KyOSK is a community-level, controlled quasi-experimental, non-randomised trial. KyOSK involves two cohorts of people who use drugs, one in an intervention county (n=425) and one in a control county (n=325). People who are 18 years or older, are community-dwelling residents in the target counties and have used drugs to get high in the past 6 months are eligible. The trial compares the effectiveness of a fixed-site, staffed syringe service programme (standard of care) with the standard of care supplemented with a kiosk. The kiosk will contain various harm reduction supplies accessible to participants upon valid code entry, allowing dispensing data to be linked to participant survey data. The kiosk will include a call-back feature that allows participants to select needed services and receive linkage-to-care services from a peer recovery coach. The cohorts complete follow-up surveys every 6 months for 36 months (three preceding kiosk implementation and four post-implementation). The study will test the effectiveness of the kiosk on reducing risk behaviours associated with overdose, HIV and hepatitis C, as well as implementation outcomes and cost-effectiveness.

ETHICS AND DISSEMINATION

The University of Kentucky Institutional Review Board approved the protocol. Results will be disseminated in academic conferences and peer-reviewed journals, online and print media, and community meetings.

TRIAL REGISTRATION NUMBER

NCT05657106.

Effects of the ARRIVE (A Randomized Trial of Induction Versus Expectant Management) Trial on Elective Induction and Obstetric Outcomes in Term Nulliparous Patients

OBJECTIVE

To evaluate the effect of publication of the ARRIVE (A Randomized Trial of Induction Versus Expectant Management) trial on perinatal outcomes in singleton, term, nulliparous patients.

METHODS

An interrupted time series analysis was performed using clinical data for nulliparous singleton births at 39 weeks of gestation or later at 13 hospitals in the Northwest region of the United States (January 2016-December 2020). A modified Poisson regression was used to model time trends and changes after the ARRIVE trial (August 9, 2018). Outcomes of interest were elective induction, unplanned cesarean births, hypertensive disorders of pregnancy, a composite of perinatal adverse outcomes, and neonatal intensive care unit admissions.

RESULTS

The analysis included 28,256 births (15,208 pre-ARRIVE and 13,048 post-ARRIVE). The rate of elective labor induction was 3.6% during the pre-ARRIVE period (January 2016-July 2018) and 10.8% post-ARRIVE (August 2018-December 2020). In the interrupted time series analysis, elective induction increased by 42% (relative risk [RR] 1.42; 95% CI 1.18-1.71) immediately after the ARRIVE trial publication. Thereafter, the trend was unchanged compared with the pre-ARRIVE period. There was no statistically significant change in cesarean birth (RR 0.96; 95% CI 0.89-1.04) or hypertensive disorders of pregnancy (RR 0.91; 95% CI 0.79-1.06) immediately after the trial, and no change in trend. After the ARRIVE trial, there was no immediate change in adverse perinatal outcomes, but a statistically significant increase in trend of adverse perinatal events (1.03; 95% CI 1.01-1.05) when compared with a declining trend observed in the pre-ARRIVE period.

CONCLUSION

Publication of the ARRIVE trial was associated with an increase in elective induction, and no change in cesarean birth or hypertensive disorders of pregnancy in singleton nulliparous patients giving birth at 39 weeks or later. There was a flattening of the pre-ARRIVE decreasing trend in perinatal adverse events.

Did the cannabis recreational use law affect traffic crash outcomes in Toronto? Building evidence for the adequate number of authorised cannabis stores' thresholds

INTRODUCTION

In the past decade, a group of studies has begun to explore the association between cannabis recreational use policies and traffic crashes. After these policies are set in place, several factors may affect cannabis consumption, including the number of cannabis stores (NCS) per capita. This study examines the association between the enactment of Canada's Cannabis Act (CCA) (18 October 2018) and the NCS (allowed to function from 1 April 2019) with traffic injuries in Toronto.

METHODS

We explored the association of the CCA and the NCS with traffic crashes. We applied two methods: hybrid difference-in-difference (DID) and hybrid-fuzzy DID. We used generalised linear models using CCA and the NCS per capita as the main variables of interest. We adjusted for precipitation, temperature and snow. Information is gathered from Toronto Police Service, Alcohol and Gaming Commission of Ontario, and Environment Canada. The period of analysis was from 1 January 2016 to 31 December 2019.

RESULTS

Regardless of the outcome, neither the CCA nor the NCS is associated with concomitant changes in the outcomes. In hybrid DID models, the CCA is associated with non-significant decreases of 9% (incidence rate ratio 0.91, 95% confidence interval 0.74,1.11) in traffic crashes and in the hybrid-fuzzy DID models, the NCS are associated with nonsignificant decreases of 3% (95% confidence interval - 9%, 4%) in the same outcome.

DISCUSSION AND CONCLUSIONS

This study observes that more research is needed to better understand the short-term effects (April to December 2019) of NCS in Toronto on road safety outcomes.

Accounting for expected attrition in the planning of cluster randomized trials for assessing treatment effect heterogeneity

BACKGROUND

Detecting treatment effect heterogeneity is an important objective in cluster randomized trials and implementation research. While sample size procedures for testing the average treatment effect accounting for participant attrition assuming missing completely at random or missing at random have been previously developed, the impact of attrition on the power for detecting heterogeneous treatment effects in cluster randomized trials remains unknown.

METHODS

We provide a sample size formula for testing for a heterogeneous treatment effect assuming the outcome is missing completely at random. We also propose an efficient Monte Carlo sample size procedure for assessing heterogeneous treatment effect assuming covariate-dependent outcome missingness (missing at random). We compare our sample size methods with the direct inflation method that divides the estimated sample size by the mean follow-up rate. We also evaluate our methods through simulation studies and illustrate them with a real-world example.

RESULTS

Simulation results show that our proposed sample size methods under both missing completely at random and missing at random provide sufficient power for assessing heterogeneous treatment effect. The proposed sample size methods lead to more accurate sample size estimates than the direct inflation method when the missingness rate is high (e.g., ≥ 30%). Moreover, sample size estimation under both missing completely at random and missing at random is sensitive to the missingness rate, but not sensitive to the intracluster correlation coefficient among the missingness indicators.

CONCLUSION

Our new sample size methods can assist in planning cluster randomized trials that plan to assess a heterogeneous treatment effect and participant attrition is expected to occur.

Implementation of a quality improvement programme using the Active Patient Link call and recall system to improve timeliness and equity of childhood vaccinations: protocol for a mixed-methods evaluation

INTRODUCTION

Call and recall systems provide actionable intelligence to improve equity and timeliness of childhood vaccinations, which have been disrupted during the COVID-19 pandemic. We will evaluate the effectiveness, fidelity and sustainability of a data-enabled quality improvement programme delivered in primary care using an Active Patient Link Immunisation (APL-Imms) call and recall system to improve timeliness and equity of uptake in a multiethnic disadvantaged urban population. We will use qualitative methods to evaluate programme delivery, focusing on uptake and use, implementation barriers and service improvements for clinical and non-clinical primary care staff, its fidelity and sustainability.

METHODS AND ANALYSIS

This is a mixed-methods observational study in 284 general practices in north east London (NEL). The target population will be preschool-aged children eligible to receive diphtheria, tetanus and pertussis (DTaP) or measles, mumps and rubella (MMR) vaccinations and registered with an NEL general practice. The intervention comprises an in-practice call and recall tool, facilitation and training, and financial incentives. The quantitative evaluation will include interrupted time Series analyses and Slope Index of Inequality. The primary outcomes will be the proportion of children receiving at least one dose of a DTaP-containing or MMR vaccination defined, respectively, as administered between age 6 weeks and 6 months or between 12 and 18 months of age. The qualitative evaluation will involve a 'Think Aloud' method and semistructured interviews of stakeholders to assess impact, fidelity and sustainability of the APL-Imms tool, and fidelity of the implementation by facilitators.

ETHICS AND DISSEMINATION

The research team has been granted permission from data controllers in participating practices to use deidentified data for audit purposes. As findings will be specific to the local context, research ethics approval is not required. Results will be disseminated in a peer-reviewed journal and to stakeholders, including parents, health providers and commissioners.

The Effect of COVID-19 on Police Activities: Traffic Stops, Arrests, and Use of Force

COVID-19 may affect police activities (i.e., traffic stops, arrests, and use of force) due to public compliance with COVID-19 restrictions, changes in individuals’ daily activities, and health threats posed by COVID-19. The purpose of this research is to examine the effects of COVID-19 on police activities and the trends in these activities in Burlington, Vermont. The data that measured arrests, traffic stops, and use of force were obtained from the Burlington Police Department website and covered the period from December 10, 2018 through June 14, 2021. Then, the daily time-series data for arrest rate, use of force rate, and traffic stop rate were created (N = 918; 459 days pre-COVID-19 and 459 days during COVID-19). The results showed that COVID-19 statistically significantly increased traffic stops and arrests, but not use of force on the day immediately following the implementation of the COVID-19 restrictions and then remained stable over time. However, no significant change was detected in the trends of arrests, traffic stops, and use of force before and during the COVID-19 period. COVID-19 had a significant immediate and lasting effect on traffic stops and arrests, but not use of force.

The use of generalized synthetic control method to evaluate air pollution control measures of G20 Hangzhou Summit

The traditional campaign-style enforcement in environmental governance has been debated whether its rebound effect is likely to eat away the short-term environmental benefits and subsequently bring about severer pollution. There are methodological challenges in assessing the effect of temporary environmental intervention. By applying the generalized synthetic control method (GSCM), we quantified and characterized the effectiveness of environmental regulations implemented for the G20 Hangzhou Summit held on 4-5 September, 2016. The summit was successful in reducing Air Quality Composite Index by 17.40% (95% CI: 9.53%, 24.60%), 13.30% (95% CI: 4.23%, 21.50%), and 10.09% (95% CI: 2.01%, 17.51%) in the core, strictly-regulated and regulated areas respectively, comparing with the index expected under a "No-G20" scenario during the preparatory period and the summit period (July-September 2016), and the reduction of the levels in specific pollutants (PM₁₀, PM_2.5, NO₂, and CO) was also observed. Besides, the environmental benefits lasted for at least 3 months after the summit. This study demonstrates that the pollution control measures during the G20 Hangzhou Summit improved air quality immediately and continuously, and the GSCM provides a useful tool for evaluating the intervention effects of environmental regulations.

Improving Medication Adherence Through Adaptive Digital Interventions (iMedA) in Patients With Hypertension: Protocol for an Interrupted Time Series Study

BACKGROUND

There is a strong need to improve medication adherence (MA) for individuals with hypertension in order to reduce long-term hospitalization costs. We believe this can be achieved through an artificial intelligence agent that helps the patient in understanding key individual adherence risk factors and designing an appropriate intervention plan. The incidence of hypertension in Sweden is estimated at approximately 27%. Although blood pressure control has increased in Sweden, barely half of the treated patients achieved adequate blood pressure levels. It is a major risk factor for coronary heart disease and stroke as well as heart failure. MA is a key factor for good clinical outcomes in persons with hypertension.

OBJECTIVE

The overall aim of this study is to design, develop, test, and evaluate an adaptive digital intervention called iMedA, delivered via a mobile app to improve MA, self-care management, and blood pressure control for persons with hypertension.

METHODS

The study design is an interrupted time series. We will collect data on a daily basis, 14 days before, during 6 months of delivering digital interventions through the mobile app, and 14 days after. The effect will be analyzed using segmented regression analysis. The participants will be recruited in Region Halland, Sweden. The design of the digital interventions follows the just-in-time adaptive intervention framework. The primary (distal) outcome is MA, and the secondary outcome is blood pressure. The design of the digital intervention is developed based on a needs assessment process including a systematic review, focus group interviews, and a pilot study, before conducting the longitudinal interrupted time series study.

RESULTS

The focus groups of persons with hypertension have been conducted to perform the needs assessment in a Swedish context. The design and development of digital interventions are in progress, and the interventions are planned to be ready in November 2020. Then, the 2-week pilot study for usability evaluation will start, and the interrupted time series study, which we plan to start in February 2021, will follow it.

CONCLUSIONS

We hypothesize that iMedA will improve medication adherence and self-care management. This study could illustrate how self-care management tools can be an additional (digital) treatment support to a clinical one without increasing burden on health care staff.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04413500; https://clinicaltrials.gov/ct2/show/NCT04413500.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/24494.

sparrpowR: a flexible R package to estimate statistical power to identify spatial clustering of two groups and its application

Background

Cancer epidemiology studies require sufficient power to assess spatial relationships between exposures and cancer incidence accurately. However, methods for power calculations of spatial statistics are complicated and underdeveloped, and therefore underutilized by investigators. The spatial relative risk function, a cluster detection technique that detects spatial clusters of point-level data for two groups (e.g., cancer cases and controls, two exposure groups), is a commonly used spatial statistic but does not have a readily available power calculation for study design.

Results

We developed sparrpowR as an open-source R package to estimate the statistical power of the spatial relative risk function. sparrpowR generates simulated data applying user-defined parameters (e.g., sample size, locations) to detect spatial clusters with high statistical power. We present applications of sparrpowR that perform a power calculation for a study designed to detect a spatial cluster of incident cancer in relation to a point source of numerous environmental emissions. The conducted power calculations demonstrate the functionality and utility of sparrpowR to calculate the local power for spatial cluster detection.

Conclusions

sparrpowR improves the current capacity of investigators to calculate the statistical power of spatial clusters, which assists in designing more efficient studies. This newly developed R package addresses a critically underdeveloped gap in cancer epidemiology by estimating statistical power for a common spatial cluster detection technique.

Revaccination with measles-mumps-rubella vaccine and hospitalization for infection in Denmark and Sweden - An interrupted time-series analysis

BACKGROUND

In a previous cohort study of 4-year-old Danish children, revaccination with the live measles-mumps-rubella vaccine (MMR) was associated with a 16% reduction in the rate of hospitalization lasting two days or longer for non-measles-mumps-rubella infections.

AIM

To examine if the introduction of revaccination with MMR at 4 years of age in Denmark (spring 2008) and at 7-9 years of age in Sweden (autumn 2009), at a time when there was virtually no measles, mumps or rubella cases, was associated with a reduction in the rate of hospitalization-for-infection lasting two days or longer at the population level.

METHODS

We included 4-year-olds in Denmark and 7-9-year-olds in Sweden. We obtained the number of hospitalization-for-infection lasting two days or longer from nationwide hospital registers. Person-years at risk were approximated from population statistics for each season and year. We performed an interrupted time series analysis using Poisson regression to estimate the change in hospitalization incidence rates following the introduction of MMR revaccination, adjusting for seasonality. We also performed analyses with control series (3-year-olds in Denmark and 4-year-olds in Sweden).

RESULTS

Comparing the incidence of hospitalization-for-infection lasting two days or longer after the introduction of MMR revaccination with the expected level without an introduction of MMR revaccination resulted in an incidence rate ratio of 1.07 (95% confidence interval [CI] = 0.89-1.28) for 4-year-olds in Denmark and 0.89 (95% CI = 0.77-1.02) for 7-9-year-olds in Sweden in analyses without controls. Analyses with controls gave similar results.

CONCLUSION

This population-level study of the introduction of MMR revaccination in Denmark and Sweden had inadequate power to confirm or refute the findings from an individual-level Danish study of an association between MMR revaccination and a lower incidence rate of hospitalization-for-infection lasting two days or longer.

Containment of Antibiotic REsistance-measures to improve antibiotic use in pregnancy, childbirth and young children (CAREChild): a protocol of a prospective, quasiexperimental interventional study in Lao PDR

INTRODUCTION

Antibiotics are essential to treat infections during pregnancy and to reduce both maternal and infant mortality. Overall use, but especially non-indicated use, and misuse of antibiotics are drivers of antibiotic resistance (ABR). High non-indicated use of antibiotics for uncomplicated vaginal deliveries is widespread in many parts of the world. Similarly, irrational use of antibiotics is reported for children. There is scarcity of evidence regarding antibiotic use and ABR in Lao PDR (Laos). The overarching aim of this project is to fill those knowledge gaps and to evaluate a quality improvement intervention. The primary objective is to estimate the proportion of uncomplicated vaginal deliveries where antibiotics are used and to compare its trend before and after the intervention.

METHODS AND ANALYSIS

This 3-year, prospective, quasiexperimental study without comparison group includes a formative and interventional phase. Data on antibiotic use during delivery will be collected from medical records. Knowledge, attitudes and reported practices on antibiotic use in pregnancy, during delivery and for children, will be collected from women through questionnaires. Healthcare providers' knowledge, attitudes and practices of antibiotics administration for pregnant women, during delivery and for children, will be collected via adapted questionnaires. Perceptions regarding antibiotics will be explored through focus group discussions with women and individual interviews with key stakeholders. Faecal samples for culturing of Escherichia coli and Klebsiella spp. and antibiotic susceptibility testing will be taken before, during and 6 months after delivery to determine colonisation of resistant strains. The planned intervention will comprise training workshops, educational materials and social media campaign and will be evaluated using interrupted time series analysis.

ETHICS AND DISSEMINATION

The project received ethical approval from the National Ethics Committee for Health Research, Ministry of Health, Laos. The results will be disseminated via scientific publications, conference presentations and communication with stakeholders.

TRAIL REGISTRATION NUMBER

ISRCTN16217522; Pre-results.

Environmental Factors and Hyperacute Stroke Care Activity During the COVID-19 Pandemic: An Interrupted Time-Series Analysis

BACKGROUND AND AIMS

Concerns have arisen regarding patient access and delivery of acute stroke care during the COVID-19 pandemic. We investigated key population level events on activity of the three hyperacute stroke units (HASUs) within Greater Manchester and East Cheshire (GM & EC), whilst adjusting for environmental factors.

METHODS

Weekly stroke admission & discharge counts in the three HASUs were collected locally from Emergency Department (ED) data and Sentinel Stroke National Audit Programme core dataset prior to, and during the emergence of the COVID-19 pandemic (Jan 2020 to May 2020). Whilst adjusting for local traffic-related air pollution and ambient measurement, an interrupted time-series analysis using a segmented generalised linear model investigated key population level events on the rate of stroke team ED assessments, admissions for stroke, referrals for transient ischaemic attack (TIA), and stroke discharges.

RESULTS

The median total number of ED stroke assessments, admissions, TIA referrals, and discharges across the three HASU sites prior to the first UK COVID-19 death were 150, 114, 69, and 76 per week. The stable weekly trend in ED assessments and stroke admissions decreased by approximately 16% (and 21% for TIAs) between first UK hospital COVID-19 death (5th March) and the implementation of the Act-FAST campaign (6th April) where a modest 4% and 5% increase per week was observed. TIA referrals increased post Government intervention (23rd March), without fully returning to the numbers observed in January and February. Trends in discharges from stroke units appeared unaffected within the study period reported here.

CONCLUSION

Despite adjustment for environmental factors stroke activity was temporarily modified by the COVID-19 pandemic. Underlying motivations within the population are still not clear. This raises concerns that patients may have avoided urgent health care risking poorer short and long-term health outcomes.

Estimating the power to detect a change caused by a vaccine from time series data

When evaluating the effects of vaccination programs, it is common to estimate changes in rates of disease before and after vaccine introduction. There are a number of related approaches that attempt to adjust for trends unrelated to the vaccine and to detect changes that coincide with introduction. However, characteristics of the data can influence the ability to estimate such a change. These include, but are not limited to, the number of years of available data prior to vaccine introduction, the expected strength of the effect of the intervention, the strength of underlying secular trends, and the amount of unexplained variability in the data. Sources of unexplained variability include model misspecification, epidemics due to unidentified pathogens, and changes in ascertainment or coding practice among others. In this study, we present a simple simulation framework for estimating the power to detect a decline and the precision of these estimates. We use real-world data from a pre-vaccine period to generate simulated time series where the vaccine effect is specified a priori. We present an interactive web-based tool to implement this approach. We also demonstrate the use of this approach using observed data on pneumonia hospitalization from the states in Brazil from a period prior to introduction of pneumococcal vaccines to generate the simulated time series. We relate the power of the hypothesis tests to the number of cases per year and the amount of unexplained variability in the data and demonstrate how fewer years of data influence the results.

Evaluating a Longitudinal Cohort of Clinics Engaging in the Family Planning Elevated Contraceptive Access Program: Study Protocol for a Comparative Interrupted Time Series Analysis

Background

Access to high-quality, comprehensive contraceptive care is an inherent component of reproductive human rights. However, hindrances to specific aspects of contraceptive provision, including availability, accessibility, acceptability, and quality, continue to perpetuate unmet needs. The state of Utah has recently passed a series of contraceptive policies intended to improve contraceptive access. Despite these positive changes to theoretical access, fiscal appropriations to support the implementation of these policies have been minimal, and many individuals still struggle to access contraception.

Objective

The Family Planning Elevated Contraceptive Access Program (FPE CAP), part of a larger statewide contraceptive initiative, specifically aims to improve contraceptive access within health clinics. This paper describes the study protocol for evaluating the success of FPE CAP.

Methods

Health clinics apply for membership in the FPE CAP. On acceptance in the program, they receive a cash grant for clinical supplies, equipment, and personnel expenses; reimbursement for contraceptive services and methods for eligible clients; technical support, training, and proctoring on counseling and providing all methods of contraception; method stocking of intrauterine devices and implants; and demand generation activities, including local media campaigns, to inform community members about the FPE CAP and possible eligibility. FPE collects monthly service delivery reports from participating clinics for evaluation purposes. The primary outcomes of FPE CAP are level and trend changes in contraceptive service delivery among individuals earning ≤138% federal poverty level (FPL) following membership in FPE CAP and among FPE CAP clients earning between 139% and 250% FPL (including those ineligible for Medicaid) compared with historical data and control clinics. To assess this, we will conduct comparative interrupted time series analyses assessing the level and trend changes in intervention and control clinics 12 months before the intervention, for the 2-year duration of the intervention, and for the subsequent 12 months following the intervention.

Results

We found that the study is adequately powered (>80% power) with our planned number of clinics and the number of months of data available in the study. To date, we have successfully completed the recruitment and enrollment of 8 of the expected 9 health organizations and 4 of the control clinics. Completed health organization enrollment for both intervention and control organizations is expected to be completed in December 2020.

Conclusions

The study aims to provide insight into a new approach to contraceptive initiatives by addressing comprehensive aspects of contraceptive care at the health system level. Ongoing state policy changes and implementation components may affect the evaluation outcomes.

International Registered Report Identifier (IRRID)

DERR1-10.2196/18308

Estimating the power to detect a change caused by a vaccine from time series data

When evaluating the effects of vaccination programs, it is common to estimate changes in rates of disease before and after vaccine introduction. There are a number of related approaches that attempt to adjust for trends unrelated to the vaccine and to detect changes that coincide with introduction. However, characteristics of the data can influence the ability to estimate such a change. These include, but are not limited to, the number of years of available data prior to vaccine introduction, the expected strength of the effect of the intervention, the strength of underlying secular trends, and the amount of unexplained variability in the data. Sources of unexplained variability include model misspecification, epidemics due to unidentified pathogens, and changes in ascertainment or coding practice among others. In this study, we present a simple simulation framework for estimating the power to detect a decline and the precision of these estimates. We use real-world data from a pre-vaccine period to generate simulated time series where the vaccine effect is specified a priori. We present an interactive web-based tool to implement this approach. We also demonstrate the use of this approach using observed data on pneumonia hospitalization from the states in Brazil from a period prior to introduction of pneumococcal vaccines to generate the simulated time series. We relate the power of the hypothesis tests to the number of cases per year and the amount of unexplained variability in the data and demonstrate how fewer years of data influence the results.