1
|
Indriani C, Tanamas SK, Khasanah U, Ansari MR, Tantowijoyo W, Ahmad RA, Dufault SM, Jewell NP, Utarini A, Simmons CP, Anders KL. Impact of randomised wmel Wolbachia deployments on notified dengue cases and insecticide fogging for dengue control in Yogyakarta City. Glob Health Action 2023; 16:2166650. [PMID: 36700745 PMCID: PMC9894080 DOI: 10.1080/16549716.2023.2166650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023] Open
Abstract
BACKGROUND Releases of Wolbachia (wMel)-infected Aedes aegypti mosquitoes significantly reduced the incidence of virologically confirmed dengue in a previous cluster randomised trial in Yogyakarta City, Indonesia. Following the trial, wMel releases were extended to the untreated control areas, to achieve city-wide coverage of Wolbachia. OBJECTIVE In this predefined analysis, we evaluated the impact of the wMel deployments in Yogyakarta on dengue hemorrhagic fever (DHF) case notifications and on the frequency of perifocal insecticide spraying by public health teams. METHODS Monthly counts of DHF cases notified to the Yogyakarta District Health Office between January 2006 and May 2022 were modelled as a function of time-varying local wMel treatment status (fully- and partially-treated vs untreated, and by quintile of wMel prevalence). The frequency of insecticide fogging in wMel-treated and untreated areas was analysed using negative binomial regression. RESULTS Notified DHF incidence was 83% lower in fully treated vs untreated periods (IRR 0.17 [95% CI 0.14, 0.20]), and 78% lower in areas with 80-100% wMel prevalence compared to areas with 0-20% wMel (IRR 0.23 [0.17, 0.30]). A similar intervention effect was observed at 60-80% wMel prevalence as at 80-100% prevalence (76% vs 78% efficacy, respectively). Pre-intervention, insecticide fogging occurred at similar frequencies in areas later randomised to wMel-treated and untreated arms of the trial. After wMel deployment, fogging occurred significantly less frequently in treated areas (IRR 0.17 [0.10, 0.30]). CONCLUSIONS Deployments of wMel-infected Aedes aegypti mosquitoes resulted in an 83% reduction in the application of perifocal insecticide spraying, consistent with lower dengue case notifications in wMel-treated areas. These results show that the Wolbachia intervention effect demonstrated previously in a cluster randomised trial was also measurable from routine surveillance data.
Collapse
Affiliation(s)
- Citra Indriani
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia,Department of Biostatistics, Epidemiology and Population Health, Faculty of Medicine Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | | | - Uswatun Khasanah
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Muhammad Ridwan Ansari
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Rubangi
- Disease Control Department, Yogyakarta City Health Office, Yogyakarta, Indonesia
| | - Warsito Tantowijoyo
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Riris Andono Ahmad
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia,Department of Biostatistics, Epidemiology and Population Health, Faculty of Medicine Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Suzanne M. Dufault
- Division of Pulmonary and Critical Care Medicine, School of Medicine, University of California, San Francisco, CA, USA
| | - Nicholas P. Jewell
- Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK
| | - Adi Utarini
- World Mosquito Program Yogyakarta, Centre for Tropical Medicine, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia,Department of Health Policy and Management, Faculty of Medicine Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | | | - Katherine L. Anders
- World Mosquito Program, Monash University, Clayton, VIC, Australia,CONTACT Katherine L. Anders World Mosquito Program, Monash University, 12 Innovation Walk, Clayton, VIC3800, Australia
| |
Collapse
|
2
|
Anglin KL, Wong VC, Wing C, Miller-Bains K, McConeghy K. The validity of causal claims with repeated measures designs: A within-study comparison evaluation of differences-in-differences and the comparative interrupted time series. EVALUATION REVIEW 2023; 47:895-931. [PMID: 37072684 DOI: 10.1177/0193841x231167672] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Modern policies are commonly evaluated not with randomized experiments but with repeated measures designs like difference-in-differences (DID) and the comparative interrupted time series (CITS). The key benefit of these designs is that they control for unobserved confounders that are fixed over time. However, DID and CITS designs only result in unbiased impact estimates when the model assumptions are consistent with the data at hand. In this paper, we empirically test whether the assumptions of repeated measures designs are met in field settings. Using a within-study comparison design, we compare experimental estimates of the impact of patient-directed care on medical expenditures to non-experimental DID and CITS estimates for the same target population and outcome. Our data come from a multi-site experiment that includes participants receiving Medicaid in Arkansas, Florida, and New Jersey. We present summary measures of repeated measures bias across three states, four comparison groups, two model specifications, and two outcomes. We find that, on average, bias resulting from repeated measures designs are very close to zero (less than 0.01 standard deviations; SDs). Further, we find that comparison groups which have pre-treatment trends that are visibly parallel to the treatment group result in less bias than those with visibly divergent trends. However, CITS models that control for baseline trends produced slightly more bias and were less precise than DID models that only control for baseline means. Overall, we offer optimistic evidence in favor of repeated measures designs when randomization is not feasible.
Collapse
Affiliation(s)
- Kylie L Anglin
- Neag School of Education, University of Connecticut, Storrs, CT, USA
| | - Vivian C Wong
- School of Education and Human Development, University of Virginia, Charlottesville, VA, USA
| | - Coady Wing
- Paul H. O'Neill School of Public and Environmental Affairs, Indiana University, Bloomington, IN, USA
| | - Kate Miller-Bains
- School of Education and Human Development, University of Virginia, Charlottesville, VA, USA
| | - Kevin McConeghy
- School of Public Health, Brown University, Providence, RI, USA
| |
Collapse
|
3
|
Miratrix LW. Using Simulation to Analyze Interrupted Time Series Designs. EVALUATION REVIEW 2022; 46:750-778. [PMID: 35634945 DOI: 10.1177/0193841x221101286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
We are sometimes forced to use the Interrupted Time Series (ITS) design as an identification strategy for potential policy change, such as when we only have a single treated unit and cannot obtain comparable controls. For example, with recent county- and state-wide criminal justice reform efforts, where judicial bodies have changed bail setting practices for everyone in their jurisdiction in order to reduce rates of pre-trial detention while maintaining court order and public safety, we have no natural and available comparison group other than the past. In these contexts, it is imperative to model pre-policy trends with a light touch, allowing for structures such as autoregressive departures from any pre-existing trend, in order to accurately and realistically assess the uncertainty of our projections. We aim to provide a methodological approach rooted in commonly understood and used modeling tools to achieve this. We quantify uncertainty with simulation, generating a distribution of plausible counterfactual trajectories to compare to the observed; this approach naturally allows for incorporating seasonality and other time-varying covariates, and provides confidence intervals along with point estimates for the potential impacts of policy change. We find simulation provides a natural framework to capture and show uncertainty in the ITS designs. It also allows for easy extensions such as nonparametric smoothing in order to handle multiple post-policy time points.
Collapse
Affiliation(s)
- Luke W Miratrix
- 80330Harvard Graduate School of Education, Cambridge, MA, USA
| |
Collapse
|
4
|
Fanshawe TR, Turner PJ, Gillespie MM, Hayward GN. The comparative interrupted time series design for assessment of diagnostic impact: methodological considerations and an example using point-of-care C-reactive protein testing. Diagn Progn Res 2022; 6:3. [PMID: 35232491 PMCID: PMC8888027 DOI: 10.1186/s41512-022-00118-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Accepted: 01/13/2022] [Indexed: 11/11/2022] Open
Abstract
BACKGROUND In diagnostic evaluation, it is necessary to assess the clinical impact of a new diagnostic as well as its diagnostic accuracy. The comparative interrupted time series design has been proposed as a quasi-experimental approach to evaluating interventions. We show how it can be used in the design of a study to evaluate a point-of-care diagnostic test for C-reactive protein in out-of-hours primary care services, to guide antibiotic prescribing among patients presenting with possible respiratory tract infection. This study consisted of a retrospective phase that used routinely collected monthly antibiotic prescribing data from different study sites, and a prospective phase in which antibiotic prescribing rates were monitored after the C-reactive protein diagnostic was introduced at some of the sites. METHODS Of 8 study sites, 3 were assigned to receive the diagnostic and 5 were assigned as controls. We obtained retrospective monthly time series of respiratory tract targeted antibiotic prescriptions at each site. Separate ARIMA models at each site were used these to forecast monthly prescription counts that would be expected in the prospective phase, using simulation to obtain a set of 1-year predictions alongside their standard errors. We show how these forecasts can be combined to test for a change in prescription rates after introduction of the diagnostic and estimate power to detect this change. RESULTS Fitted time series models at each site were stationary and showed second-order annual seasonality, with a clear December peak in prescriptions, although the timing and extent of the peak varied between sites and between years. Mean one-year predictions of antibiotic prescribing rates based on the retrospective time series analysis differed between sites assigned to receive the diagnostic and those assigned to control. Adjusting for the trend in the retrospective time series at each site removed these differences. CONCLUSIONS Quasi-experimental designs such as comparative interrupted time series can be used in diagnostic evaluation to estimate effect sizes before conducting a full randomised controlled trial or if a randomised trial is infeasible. In multi-site studies, existing retrospective data should be used to adjust for underlying differences between sites to make outcome data from different sites comparable, when possible.
Collapse
Affiliation(s)
- Thomas R Fanshawe
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, OX2 6GG, UK.
| | - Philip J Turner
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, OX2 6GG, UK
| | - Marjorie M Gillespie
- Practice Plus Group, Hawker House, 5-6 Napier Court, Napier Road, Reading, Berkshire, England, RG1 8BW, UK
| | - Gail N Hayward
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, OX2 6GG, UK
| |
Collapse
|
5
|
Hill T, Coupland C, Kendrick D, Jones M, Akbari A, Rodgers S, Watson MC, Tyrrell E, Merrill S, Orton E. Impact of the national home safety equipment scheme 'Safe At Home' on hospital admissions for unintentional injury in children under 5: a controlled interrupted time series analysis. J Epidemiol Community Health 2022; 76:53-59. [PMID: 34158405 PMCID: PMC8666806 DOI: 10.1136/jech-2021-216613] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Accepted: 05/31/2021] [Indexed: 12/26/2022]
Abstract
BACKGROUND Unintentional home injuries are a leading cause of preventable death in young children. Safety education and equipment provision improve home safety practices, but their impact on injuries is less clear. Between 2009 and 2011, a national home safety equipment scheme was implemented in England (Safe At Home), targeting high-injury-rate areas and socioeconomically disadvantaged families with children under 5. This provided a 'natural experiment' for evaluating the scheme's impact on hospital admissions for unintentional injuries. METHODS Controlled interrupted time series analysis of unintentional injury hospital admission rates in small areas (Lower Layer Super Output Areas (LSOAs)) in England where the scheme was implemented (intervention areas, n=9466) and matched with LSOAs in England and Wales where it was not implemented (control areas, n=9466), with subgroup analyses by density of equipment provision. RESULTS 57 656 homes receiving safety equipment were included in the analysis. In the 2 years after the scheme ended, monthly admission rates declined in intervention areas (-0.33% (-0.47% to -0.18%)) but did not decline in control areas (0.04% (-0.11%-0.19%), p value for difference in trend=0.001). Greater reductions in admission rates were seen as equipment provision density increased. Effects were not maintained beyond 2 years after the scheme ended. CONCLUSIONS A national home safety equipment scheme was associated with a reduction in injury-related hospital admissions in children under 5 in the 2 years after the scheme ended. Providing a higher number of items of safety equipment appears to be more effective in reducing injury rates than providing fewer items.
Collapse
Affiliation(s)
- Trevor Hill
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Carol Coupland
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Denise Kendrick
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Matthew Jones
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | | | - Sarah Rodgers
- Public Health and Policy, University of Liverpool, Liverpool, UK
| | | | - Edward Tyrrell
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Sheila Merrill
- Royal Society for the Prevention of Accidents (RoSPA), Edgbaston, UK
| | - Elizabeth Orton
- Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| |
Collapse
|
6
|
Langenberger B, Baier N, Hanke FC, Fahrentholz J, Gorny C, Sehlen S, Reber KC, Liersch S, Radomski R, Haftenberger J, Heppner HJ, Busse R, Vogt V. The detection and prevention of adverse drug events in nursing home and home care patients: Study protocol of a quasi-experimental study. Nurs Open 2021; 9:1477-1485. [PMID: 34859616 PMCID: PMC8859083 DOI: 10.1002/nop2.1146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Revised: 10/21/2021] [Accepted: 11/16/2021] [Indexed: 11/11/2022] Open
Abstract
AIM To estimate the cost-effectiveness of an intervention facilitating the early detection of adverse drug events through the means of health professional training and the application of a digital screening tool. DESIGN Multi-centred non-randomized controlled trial from August 2018 to March 2020 including 65 nursing homes or home care providers. METHODS We aim to estimate the effect of the intervention on the rate of adverse drug events as primary outcome through a quasi-experimental empirical study design. As secondary outcomes, we use hospital admissions and falls. All outcomes will be measured on patient-month level. Once the causal effect of the intervention is estimated, cost-effectiveness will be calculated. For cost-effectiveness, we include all patient costs observed by the German statutory health insurance. RESULTS The results of this study will inform about the cost-effectiveness of the optimized drug supply intervention and provide evidence for potential reimbursement within the German statutory health insurance system.
Collapse
Affiliation(s)
- Benedikt Langenberger
- Department of Health Care Management, Technische Universität Berlin, Berlin, Germany
| | - Natalie Baier
- Department of Global Health Economics, Kiel Institute for the World Economy, Kiel, Germany
| | | | - Jacqueline Fahrentholz
- Long-term Care Services Management, AOK Nordost - Die Gesundheitskasse, Potsdam, Germany
| | - Christina Gorny
- Long-term Care Services Management, AOK Nordost - Die Gesundheitskasse, Potsdam, Germany
| | - Stephanie Sehlen
- Health Services Management, AOK Nordost - Die Gesundheitskasse, Potsdam, Germany
| | | | - Sebastian Liersch
- Health Services Management, AOK Nordost - Die Gesundheitskasse, Potsdam, Germany
| | - Ralf Radomski
- Department of Innovative Health Services, VIACTIV Krankenkasse, Bochum, Germany
| | - Jens Haftenberger
- Department of Contract Management, IKK Brandenburg und Berlin, Potsdam, Germany
| | - Hans Jürgen Heppner
- Department of Geriatrics, Universität Witten/Herdecke, Witten, Germany.,Clinic of Geriatrics, Helios Klinikum Schwelm, Schwelm, Germany
| | - Reinhard Busse
- Department of Health Care Management, Technische Universität Berlin, Berlin, Germany
| | - Verena Vogt
- Department of Health Care Management, Technische Universität Berlin, Berlin, Germany
| |
Collapse
|
7
|
Kuunibe N, Lohmann J, Hillebrecht M, Nguyen HT, Tougri G, De Allegri M. What happens when performance-based financing meets free healthcare? Evidence from an interrupted time-series analysis. Health Policy Plan 2021; 35:906-917. [PMID: 32601671 DOI: 10.1093/heapol/czaa062] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/11/2020] [Indexed: 11/12/2022] Open
Abstract
In spite of the wide attention performance-based financing (PBF) has received over the past decade, no evidence is available on its impacts on quantity and mix of service provision nor on its interaction with parallel health financing interventions. Our study aimed to examine the PBF impact on quantity and mix of service provision in Burkina Faso, while accounting for the parallel introduction of a free healthcare policy. We used Health Management Information System data from 838 primary-level health facilities across 24 districts and relied on an interrupted time-series analysis with independent controls. We placed two interruptions, one to account for PBF and one to account for the free healthcare policy. In the period before the free healthcare policy, PBF produced significant but modest increases across a wide range of maternal and child services, but a significant decrease in child immunization coverage. In the period after the introduction of the free healthcare policy, PBF did not affect service provision in intervention compared with control facilities, possibly indicating a saturation effect. Our findings indicate that PBF can produce modest increases in service provision, without altering the overall service mix. Our findings, however, also indicate that the introduction of other health financing reforms can quickly crowd out the effects produced by PBF. Further qualitative research is required to understand what factors allow healthcare providers to increase the provision of some, but not all services and how they react to the joint implementation of PBF and free health care.
Collapse
Affiliation(s)
- Naasegnibe Kuunibe
- IHeidelberg Institute of Global Health, University Hospital and Medical Faculty, Heidelberg University, Im Neuenheimer Feld 365, 69120 Heidelberg, Germany.,Department of Economics and Entrepreneurship Development, Faculty of Integrated development Studies, University for Development Studies, Wa Campus, Box 520, Wa, Upper West Region, Ghana
| | - Julia Lohmann
- IHeidelberg Institute of Global Health, University Hospital and Medical Faculty, Heidelberg University, Im Neuenheimer Feld 365, 69120 Heidelberg, Germany.,Department of Global Health and Development, London School of Hygiene & Tropical Medicine, London, UK
| | - Michael Hillebrecht
- Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), Sectoral Department, Dag-Hammarskjöld-Weg 1-5, 65760 Eschborn, Germany
| | - Hoa Thi Nguyen
- IHeidelberg Institute of Global Health, University Hospital and Medical Faculty, Heidelberg University, Im Neuenheimer Feld 365, 69120 Heidelberg, Germany
| | | | - Manuela De Allegri
- IHeidelberg Institute of Global Health, University Hospital and Medical Faculty, Heidelberg University, Im Neuenheimer Feld 365, 69120 Heidelberg, Germany
| |
Collapse
|
8
|
Abstract
OBJECTIVES Initial evidence suggests that state-level regulatory mandates for sepsis quality improvement are associated with decreased sepsis mortality. However, sepsis mandates require financial investments on the part of hospitals and may lead to increased spending. We evaluated the effects of the 2013 New York State sepsis regulations on the costs of care for patients hospitalized with sepsis. DESIGN Retrospective cohort study using state discharge data from the U.S. Healthcare Costs and Utilization Project and a comparative interrupted time series analytic approach. Costs were calculated from admission-level charge data using hospital-specific cost-to-charge ratios. SETTING General, short stay, acute care hospitals in New York, and four control states: Florida, Massachusetts, Maryland, and New Jersey. PATIENTS All patients hospitalized with sepsis between January 1, 2011, and September 30, 2015. INTERVENTIONS The 2013 New York mandate that all hospitals develop and implement protocols for sepsis identification and treatment, educate staff, and report performance data to the state. MEASUREMENTS AND MAIN RESULTS The analysis included 1,026,664 admissions in 520 hospitals. Mean unadjusted costs per hospitalization in New York State were $42,036 ± $60,940 in the pre-regulation period and $39,719 ± $59,063 in the post-regulation period, compared with $34,642 ± $52,403 pre-regulation and $31,414 ± $48,155 post-regulation in control states. In the comparative interrupted time series analysis, the regulations were not associated with a significant difference in risk-adjusted mean cost per hospitalization (p = 0.12) or risk-adjusted mean cost per hospital day (p = 0.44). For example, in the 10th quarter after implementation of the regulations, risk-adjusted mean cost per hospitalization was $3,627 (95% CI, -$681 to $7,934) more than expected in New York State relative to control states. CONCLUSIONS Mandated protocolized sepsis care was not associated with significant changes in hospital costs in patients hospitalized with sepsis in New York State.
Collapse
|
9
|
Harding DJ, Sanbonmatsu L, Duncan GJ, Gennetian LA, Katz LF, Kessler RC, Kling JR, Sciandra M, Ludwig J. EVALUATING CONTRADICTORY EXPERIMENTAL AND NON-EXPERIMENTAL ESTIMATES OF NEIGHBORHOOD EFFECTS ON ECONOMIC OUTCOMES FOR ADULTS. HOUSING POLICY DEBATE 2021; 33:453-486. [PMID: 37347089 PMCID: PMC10281691 DOI: 10.1080/10511482.2021.1881985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2020] [Accepted: 01/24/2021] [Indexed: 06/23/2023]
Abstract
Although non-experimental studies find robust neighborhood effects on adults, such findings have been challenged by results from the Moving to Opportunity (MTO) residential mobility experiment. Using a within-study comparison design, this paper compares experimental and non-experimental estimates from MTO and a parallel analysis of the Panel Study of Income Dynamics (PSID). Striking similarities were found between non-experimental estimates based on MTO and PSID. No clear evidence was found that different estimates are related to duration of adult exposure to disadvantaged neighborhoods, non-linear effects of neighborhood conditions, magnitude of the change in neighborhood context, frequency of moves, treatment effect heterogeneity, or measurement, although uncertainty bands around our estimates were sometimes large. One other possibility is that MTO-induced moves might have been unusually disruptive, but results are inconsistent for that hypothesis. Taken together, the findings suggest that selection bias might account for evidence of neighborhood effects on adult economic outcomes in non-experimental studies.
Collapse
Affiliation(s)
- David J Harding
- University of California at Berkeley, Department of Sociology, 462 Barrows Hall, Berkeley CA 94720
| | - Lisa Sanbonmatsu
- Center for Education Policy Research, Harvard University, 50 Church Street, 4th Floor, Cambridge, MA, 02138
| | - Greg J Duncan
- University of California, Irvine, 2056 Education Building, Mail code 5500, Irvine, CA 92697
| | - Lisa A Gennetian
- Duke University, Sanford School of Public Policy, 212 Rubenstein Hall, 302 Towerview Road, Durham, NC 27708
| | - Lawrence F Katz
- Harvard University, Department of Economics, Cambridge, MA 02138 and NBER
| | - Ronald C Kessler
- Harvard Medical School, Department of Health Care Policy, 180 Longwood Avenue, Boston, MA 02115
| | - Jeffrey R Kling
- Congressional Budget Office, 2nd & D Streets, SW, Washington, DC 20515 and NBER
| | - Matthew Sciandra
- RTI International, 307 Waverley Oaks Rd, #101, Waltham, MA 02452
| | - Jens Ludwig
- University of Chicago, 1155 East 60th Street, Chicago, IL 60637
| |
Collapse
|
10
|
Gigli KH, Davis BS, Yabes JG, Chang CCH, Angus DC, Hershey TB, Marin JR, Martsolf GR, Kahn JM. Pediatric Outcomes After Regulatory Mandates for Sepsis Care. Pediatrics 2020; 146:peds.2019-3353. [PMID: 32605994 PMCID: PMC7329251 DOI: 10.1542/peds.2019-3353] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/24/2020] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND In 2013, New York introduced regulations mandating that hospitals develop pediatric-specific protocols for sepsis recognition and treatment. METHODS We used hospital discharge data from 2011 to 2015 to compare changes in pediatric sepsis outcomes in New York and 4 control states: Florida, Massachusetts, Maryland, and New Jersey. We examined the effect of the New York regulations on 30-day in-hospital mortality using a comparative interrupted time-series approach, controlling for patient and hospital characteristics and preregulation temporal trends. RESULTS We studied 9436 children admitted to 237 hospitals. Unadjusted pediatric sepsis mortality decreased in both New York (14.0% to 11.5%) and control states (14.4% to 11.2%). In the primary analysis, there was no significant effect of the regulations on mortality trends (differential quarterly change in mortality in New York compared with control states: -0.96%; 95% confidence interval [CI]: -1.95% to 0.02%; P = .06). However, in a prespecified sensitivity analysis excluding metropolitan New York hospitals that participated in earlier sepsis quality improvement, the regulations were associated with improved mortality trends (differential change: -2.08%; 95% CI: -3.79% to -0.37%; P = .02). The regulations were also associated with improved mortality trends in several prespecified subgroups, including previously healthy children (differential change: -1.36%; 95% CI: -2.62% to -0.09%; P = .04) and children not admitted through the emergency department (differential change: -2.42%; 95% CI: -4.24% to -0.61%; P = .01). CONCLUSIONS Implementation of statewide sepsis regulations was generally associated with improved mortality trends in New York State, particularly in prespecified subpopulations of patients, suggesting that the regulations were successful in affecting sepsis outcomes.
Collapse
Affiliation(s)
- Kristin H. Gigli
- Clinical Research, Investigation, and Systems Modeling of Acute Illness Center, Department of Critical Care Medicine
| | - Billie S. Davis
- Clinical Research, Investigation, and Systems Modeling of Acute Illness Center, Department of Critical Care Medicine
| | - Jonathan G. Yabes
- Division of General Internal Medicine, Department of Medicine,,Departments of Biostatistics and
| | - Chung-Chou H. Chang
- Division of General Internal Medicine, Department of Medicine,,Departments of Biostatistics and,Health Policy and Management, Graduate School of Public Health, and
| | - Derek C. Angus
- Clinical Research, Investigation, and Systems Modeling of Acute Illness Center, Department of Critical Care Medicine,,Health Policy and Management, Graduate School of Public Health, and
| | | | - Jennifer R. Marin
- Division of Pediatric Emergency Medicine, Department of Pediatrics, School of Medicine
| | - Grant R. Martsolf
- Department of Acute and Tertiary Care, School of Nursing, University of Pittsburgh, Pittsburgh, Pennsylvania; and,RAND Corporation, Pittsburgh, Pennsylvania
| | - Jeremy M. Kahn
- Clinical Research, Investigation, and Systems Modeling of Acute Illness Center, Department of Critical Care Medicine,,Health Policy and Management, Graduate School of Public Health, and
| |
Collapse
|
11
|
Kahn JM, Davis BS, Yabes JG, Chang CCH, Chong DH, Hershey TB, Martsolf GR, Angus DC. Association Between State-Mandated Protocolized Sepsis Care and In-hospital Mortality Among Adults With Sepsis. JAMA 2019; 322:240-250. [PMID: 31310298 PMCID: PMC6635905 DOI: 10.1001/jama.2019.9021] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
IMPORTANCE Beginning in 2013, New York State implemented regulations mandating that hospitals implement evidence-based protocols for sepsis management, as well as report data on protocol adherence and clinical outcomes to the state government. The association between these mandates and sepsis outcomes is unknown. OBJECTIVE To evaluate the association between New York State sepsis regulations and the outcomes of patients hospitalized with sepsis. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study of adult patients hospitalized with sepsis in New York State and in 4 control states (Florida, Maryland, Massachusetts, and New Jersey) using all-payer hospital discharge data (January 1, 2011-September 30, 2015) and a comparative interrupted time series analytic approach. EXPOSURES Hospitalization for sepsis before (January 1, 2011-March 31, 2013) vs after (April 1, 2013-September 30, 2015) implementation of the 2013 New York State sepsis regulations. MAIN OUTCOMES AND MEASURES The primary outcome was 30-day in-hospital mortality. Secondary outcomes were intensive care unit admission rates, central venous catheter use, Clostridium difficile infection rates, and hospital length of stay. RESULTS The final analysis included 1 012 410 sepsis admissions to 509 hospitals. The mean age was 69.5 years (SD, 16.4 years) and 47.9% were female. In New York State and in the control states, 139 019 and 289 225 patients, respectively, were admitted before implementation of the sepsis regulations and 186 767 and 397 399 patients, respectively, were admitted after implementation of the sepsis regulations. Unadjusted 30-day in-hospital mortality was 26.3% in New York State and 22.0% in the control states before the regulations, and was 22.0% in New York State and 19.1% in the control states after the regulations. Adjusting for patient and hospital characteristics as well as preregulation temporal trends and season, mortality after implementation of the regulations decreased significantly in New York State relative to the control states (P = .02 for the joint test of the comparative interrupted time series estimates). For example, by the 10th quarter after implementation of the regulations, adjusted absolute mortality was 3.2% (95% CI, 1.0% to 5.4%) lower than expected in New York State relative to the control states (P = .004). The regulations were associated with no significant differences in intensive care unit admission rates (P = .09) (10th quarter adjusted difference, 2.8% [95% CI, -1.7% to 7.2%], P = .22), a significant relative decrease in hospital length of stay (P = .04) (10th quarter adjusted difference, 0.50 days [95% CI, -0.47 to 1.47 days], P = .31), a significant relative decrease in the C difficile infection rate (P < .001) (10th quarter adjusted difference, -1.8% [95% CI, -2.6% to -1.0%], P < .001), and a significant relative increase in central venous catheter use (P = .02) (10th quarter adjusted difference, 4.8% [95% CI, 2.3% to 7.4%], P < .001). CONCLUSIONS AND RELEVANCE In New York State, mandated protocolized sepsis care was associated with a greater decrease in sepsis mortality compared with sepsis mortality in control states that did not implement sepsis regulations. Because baseline mortality rates differed between New York and comparison states, it is uncertain whether these findings are generalizable to other states.
Collapse
Affiliation(s)
- Jeremy M. Kahn
- CRISMA Center, Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
- Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Billie S. Davis
- CRISMA Center, Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Jonathan G. Yabes
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
- Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Chung-Chou H. Chang
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
- Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - David H. Chong
- Division of Pulmonary, Allergy and Critical Care Medicine, Columbia University Medical Center, New York, New York
| | - Tina Batra Hershey
- Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Grant R. Martsolf
- Department of Acute and Tertiary Care, School of Nursing, University of Pittsburgh, Pittsburgh, Pennsylvania
- RAND Corporation, Pittsburgh, Pennsylvania
| | - Derek C. Angus
- CRISMA Center, Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
- Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|
12
|
Witvorapong N, Yakubu KY. Effectiveness of antimalarial interventions in Nigeria: Evidence from facility-level longitudinal data. Health Serv Res 2019; 54:669-677. [PMID: 30740696 DOI: 10.1111/1475-6773.13122] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
OBJECTIVE To evaluate the effectiveness of a program of antimalarial interventions implemented in 2010-2013 in Niger State, Nigeria. DATA SOURCES Utilization reports from 99 intervention and 51 non-intervention health facilities from the Niger State Malaria Elimination Program, supplemented by data on facility-level characteristics from the Niger State Primary Health Care Development Agency and Local Government Malaria Control units. STUDY DESIGN Estimated with mixed-effects negative binomial modeling, a difference-in-differences method was used to quantify the impact of the program on the number of febrile illness cases and confirmed malaria cases. Potential confounding factors, non-stationarity, seasonality, and autocorrelation were explicitly accounted for. DATA EXTRACTION METHODS Data were retrieved from hard copies of utilization reports and manually inputted to create a panel of 5550 facility-month observations. PRINCIPAL FINDINGS The program was implemented in two phases. The first phase (August 2010-June 2012) involved the provision of free artemisinin-based combination therapies, long-lasting insecticidal nets, and intermittent preventive treatments. In the second phase (July 2012-March 2013), the program introduced an additional intervention: free parasite-based rapid diagnostic tests. Compared to the pre-intervention period, the average number of monthly febrile illness and malaria cases increased by 20.876 (P < 0.01) and 22.835 (P < 0.01) in the first phase, and by 19.007 (P < 0.05) and 19.681 (P < 0.05) in the second phase, respectively. The results are consistent across different evaluation methods. CONCLUSIONS This study suggests that user-fee removal leads to increased utilization of antimalarial services. It motivates future studies to cautiously select their investigative methods.
Collapse
Affiliation(s)
- Nopphol Witvorapong
- Center for Health Economics, Faculty of Economics, Chulalongkorn University, Pathumwan, Bangkok, Thailand
| | - Kolo Yaro Yakubu
- Strengthening Accountability and Quality Improvement for Maternal, Newborn and Child Health Project, Pact Nigeria, Gombe, Nigeria
| |
Collapse
|
13
|
Wong VC, Steiner PM, Anglin KL. What Can Be Learned From Empirical Evaluations of Nonexperimental Methods? EVALUATION REVIEW 2018; 42:147-175. [PMID: 30317886 DOI: 10.1177/0193841x18776870] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Given the widespread use of nonexperimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the same target population. The goal is to assess whether the stringent assumptions required for NE methods are likely to be met in practice. This essay provides an overview of recent efforts to empirically evaluate NE method performance in field settings. We discuss a brief history of the design, highlighting methodological innovations along the way. We also describe papers that are included in this two-volume special issue on WSC approaches and suggest future areas for consideration in the design, implementation, and analysis of WSCs.
Collapse
Affiliation(s)
- Vivian C Wong
- 1 Curry School of Education, University of Virginia, Charlottesville, VA, USA
| | - Peter M Steiner
- 2 Department of Educational Psychology, University of Wisconsin-Madison, Madison, WI, USA
| | - Kylie L Anglin
- 1 Curry School of Education, University of Virginia, Charlottesville, VA, USA
| |
Collapse
|