Composite measures of quality of health care: Evidence mapping of methodology and reporting

A holistic evaluation of sexual health disease investigation: case study of the 2022 Mpox outbreak in Santa Clara County, California

Background Despite its integral role in preventing and controlling infectious diseases, there is limited research that evaluates the impact of disease investigation (DI) services. The County of Santa Clara Public Health Department activated its emergency response operations, which included designated DI services, to respond to its local Mpox outbreak. The aim of this evaluation was to understand the range of outcomes achieved through Mpox DI services. Methods Mpox investigations completed between June and December 2022 were included in an evaluation that employed a composite measure to calculate the number of investigations needed to achieve the following outcomes: (1) treatment completion, (2) monitoring completion, (3) partial vaccine dose completion, (4) full vaccination series completion, and (5) STI and HIV testing. Results The overall composite score of 0.53 indicated that each investigation led to 1.90 outcomes achieved. Among cases eligible for treatment, 2.16 investigations yielded one treatment completion; 1.19 case and contact investigations yielded one monitoring completion; 2.21 and 3.53 contact investigations yielded one partial vaccine dose completion and one full vaccination series completion, respectively; and 2.25 case and contact investigations yielded one STI or HIV test. Conclusion Recognizing the multiple steps involved in DI can inform holistic evaluations that illuminate intervention impact.

The effectiveness of a multi-domain electronic feedback report on the performance of quality indicators for chronic conditions: Protocol for a randomized controlled trial in general practice

BACKGROUND

Chronic conditions are a significant public health concern due to their rising prevalence, association with high mortality, and substantial healthcare costs. General practitioners play a crucial role in managing these conditions, and quality indicators are essential tools for assessing the quality of care. Electronic feedback reports incorporating quality indicator performance have shown promise in improving care quality. However, most studies have focused on single conditions or link feedback to financial incentives, which may not sustain long-term practice changes. This study aims to evaluate the effectiveness of a multi-condition electronic feedback reports on quality indicator performance in Swiss general practice without financial incentives.

METHODS

This randomized controlled trial involves general practitioners enrolled in the FIRE project, a database of electronic medical records from Swiss primary care. Participants are randomized to receive either a plain feedback report or a comprehensive quality indicator -specific feedback report bi-monthly for 12 months. The plain feedback report contains descriptive summaries of practice activities, while the quality indicator-specific feedback report includes performance data on 14 quality indicators across cardiovascular, endocrine, pulmonary, and renal domains. The quality indicators were selected in multi-step process involving review of the literature and clinical guidelines, domain expert consultations, and a panel discussion with general practitioners. The primary study objective is to compare the effectiveness of the quality indicator-specific feedback report and of the plain feedback report with respect to the performance of the selected quality indicators.

CONCLUSION

The study addresses a critical gap by evaluating a multi-condition feedback report without financial incentives. Its findings can inform future health policies and strategies, in line with national and international initiatives that promote or even require the implementation of quality measurement activities in general practice.

TRIAL REGISTRATION

Trial registry: ISRCTN. Registration number: ISRCTN10637092, https://www.isrctn.com/ISRCTN10637092. Registered January 9, 2024.

Practices for Reporting Scale Structure and Summarizing Scores in Studies Using FAMCARE Scale to Assess Caregiver Satisfaction with Cancer Care: A Scoping Review

Background

Satisfaction with care is a concept quantified through diverse measurement tools. However, studies have indicated that measuring satisfaction is challenging due to the construct's multidimensional expression. Thus, obtaining valid results requires careful consideration of the construct's nature and measurement methods.

Purpose

The primary aim of this study was to examine how studies involving cancer caregivers have addressed the dimensionality of the construct when using satisfaction with care as an outcome, and whether this is reflected in the score reporting practices. We chose to investigate this by conducting a scoping review of the measurement tool Family Satisfaction with End-of-Life Care Scale (FAMCARE Scale), where scores can be reported as the mean of overall score, subscale scores and single-item scores.

Methods

This scoping review consisted of systematic searches using Medline, CINAHL, Embase, PsycInfo, Cochrane Library, and Epistemonikos. Two researchers used the Rayyan Qatar Computing Research Institute system to perform a blinded screening process. We extracted information on study design, purpose, evaluating of structural validity, variations in the type of scores reported, and justification for choosing the type(s) of scores that were analyzed.

Results

Twenty-three studies were included in the review, and their designs and reporting practices of score type varied substantially. Five studies reported analyses to test the scale's structural validity. Ten studies provided a justification for their choice of reporting method. The most common reporting practice found was using mean of overall scores, present in 20 of the included studies. Twelve studies reported mean of subscale scores, and ten reported single-item mean scores.

Conclusion

We found substantial variability in score reporting practices, highlighting the need for a more in-depth understanding and reflection on the multidimensional nature of caregiver satisfaction.

One for all? Assessing the quality of Italian hospital care with the "benefit of the doubt" composite indicator methods

Quality assessment in healthcare systems is challenging due to the multidimensional nature of healthcare services. This study evaluates the overall quality provided by hospitals using composite indicators under the Benefit of the Doubt (BoD) approach, which determines the weights of the indicators with minimal assumptions. We used data from 2015-2020 for Italian Local Health Authorities (LHAs) for 21 outcome measures, applying various non-parametric methods to address aggregation and weighting challenges. Our results show that the BoD measures are robust and effectively capture the dynamics of the quality of LHA, even during external shocks such as the COVID-19 pandemic. This research highlights the importance of methodological choices in the construction of composite indicators and demonstrates the effectiveness of the BoD approach in providing a comprehensive measure of healthcare quality.

Process indicators outshine outcome measures: assessing hospital quality of care in breast cancer treatment in China

Reporting the results of quality indicators can narrow the gap in the quality of care between hospitals. While most studies rely on outcome indicators, they may not accurately measure the quality of care. Process indicators are not only strongly associated with treatment outcomes, but are also more sensitive to whether patients are treated accurately, enabling timely intervention. Our study aims to investigate whether process indicators provide a more reasonable assessment of hospital quality of care compared to outcome indicators. Data were sourced from the Specific Disease Medical Service Quality Management and Control System in China. A total of 113,942 patients with breast cancer treated in 298 hospitals between January 2019 and April 2023 were included in this retrospective study. The rankability of 11 process indicators was calculated and used as a weight to create a new composite indicator. The composite indicators and outcome measures were compared using the O/E ratio categories. Finally, in order to determine the impact of different years on the results, a sensitivity analysis was conducted using bootstrap sampling. The rankability ( ρ ) values of the eleven process indicators showed significant differences, with the highest ρ value for preoperative cytological or histological examination before surgery (0.919). The ρ value for the outcome indicator was 0.011. The rankability-weighting method yielded a comprehensive score ( ρ  = 0.883). The comparison with categorical results of the outcome indicator has different performance classifications for 113 hospitals (37.92%) for composite scores and 140 (46.98%) for preoperative cytological or histological examinationbefore surgery. Process indicators are more suitable than outcome indicators for assessing the quality of breast cancer care in hospitals. Healthcare providers can use process indicators to identify specific areas for improvement, thereby driving continuous quality improvement efforts.

Quality of early stroke care and long-term mortality in patients with acute stroke: A nationwide follow-up study

INTRODUCTION

High quality of early stroke care is essential for optimizing the chance of a good patient outcome. The quality of care may be monitored by process performance measures (PPMs) and previous studies have found an association between fulfilment of PPMs and short-term mortality. However, the association with long-term mortality remains to be determined. We aimed to evaluate the association between fulfilment of PPMs and long-term mortality for patients with acute stroke in Denmark.

PATIENTS AND METHODS

We used data from Danish health care registers between 2008 and 2020 to identify all patients admitted with incident stroke (haemorrhagic (ICH) or ischaemic stroke). The quality of early stroke care was assessed using 10 PPMs. Mortality was compared using Cox proportional hazard ratios, risk ratios computed using Poisson regression, and standardized relative survival.

RESULTS

We included 102,742 patients; 9804 cases of ICH, 88,591 cases of ischaemic stroke, and 4347 cases of unspecified strokes. The cumulative 10-year mortality risk was 56.8%. Fulfilment of the individual PPMs was associated with adjusted hazard rate ratios of death between 0.76 and 0.96. Patients with 100% fulfilment of all PPMs had a lower 10-year post-stroke mortality (adjusted risk ratio 0.90) compared to the patients with 0%-49% fulfilment and a standardized relative survival of 81.3%, compared to the general population.

CONCLUSION

High quality of early stroke care was associated with lower long-term mortality following both ICH and ischaemic stroke, which emphasizes the importance of continued attention on the ability of stroke care providers to deliver high quality of early care.

Selecting patient-reported outcome measures: "what" and "for whom"

Patient-reported outcomes (PROMs) are becoming more widely implemented across health care for important reasons. However, with thousands of PROMs available and the science of psychometrics becoming more widely applied in health measurement, choosing the right ones to implement can be puzzling. This article provides a framework of the different types of PROMs by organizing them into 4 categories based upon "what" is being measured and "from whom" the questions are asked: (1) condition-specific and domain-specific, (2) condition-specific and global, (3) universal and global, and (4) universal and domain-specific. We delve deeper into each category with clinical examples. This framework can empower health care leaders and policymakers to make more informed decisions when selecting the best PROMs to implement, ensuring PROMs deliver on their potential to promote high quality, patient-centered care.

Prevalence and impact of SARS-CoV-2 infection among patients with acute ischaemic stroke: a nationwide register-based cohort study in Denmark

OBJECTIVES

An increased risk of stroke has been reported among patients with COVID-19 caused by SARS-CoV-2. We aimed to investigate the nationwide prevalence of SARS-CoV-2 among patients with acute ischaemic stroke and to study the impact on stroke severity, quality of care and mortality on an individual patient level.

DESIGN

This was a nationwide register-based cohort study.

SETTING

We used data from several Danish registers which were linked at an individual patient level using the unique civil registration number assigned to all Danish citizens. Patients were identified from the Danish Stroke Registry and information on SARS-CoV-2 infection status was collected from the Danish National COVID-19 Registry. Concurrent SARS-CoV-2 infection was defined as a positive PCR test within 31 days prior to, and 1 day after, stroke admission. Information on comorbidity was collected from the Danish National Patient Registry and information on vital status was collected from the Danish Civil Registration System.

PARTICIPANTS

A total of 11 502 patients admitted with acute ischaemic stroke from 10 March 2020 to 31 May 2021 were included in the study.

RESULTS

Among the included patients, the majority (84.6%) were tested for SARS-CoV-2, but only 68 had a positive test. These patients were more prone to have atrial fibrillation and were more often treated with reperfusion therapy. They had a significantly increased risk of severe stroke (adjusted relative risk (aRR) 1.93, 95% CI: 1.22 to 3.04) and a significantly increased 30-day mortality risk (aRR 2.29, 95% CI: 1.19 to 4.39). There was no difference in the proportion of patients fulfilling relevant performance measures on quality of care.

CONCLUSION

In this nationwide study, only 0.6% of patients with acute ischaemic stroke were tested positive for a concurrent SARS-CoV-2 infection. The patients with SARS-CoV-2 presented with more severe strokes.

Biases in Electronic Health Records Data for Generating Real-World Evidence: An Overview

Electronic Health Records (EHR) are increasingly being perceived as a unique source of data for clinical research as they provide unprecedentedly large volumes of real-time data from real-world settings. In this review of the secondary uses of EHR, we identify the anticipated breadth of opportunities, pointing out the data deficiencies and potential biases that are likely to limit the search for true causal relationships. This paper provides a comprehensive overview of the types of biases that arise along the pathways that generate real-world evidence and the sources of these biases. We distinguish between two levels in the production of EHR data where biases are likely to arise: (i) at the healthcare system level, where the principal source of bias resides in access to, and provision of, medical care, and in the acquisition and documentation of medical and administrative data; and (ii) at the research level, where biases arise from the processes of extracting, analyzing, and interpreting these data. Due to the plethora of biases, mainly in the form of selection and information bias, we conclude with advising extreme caution about making causal inferences based on secondary uses of EHRs.

Methods to derive composite indicators used for quality and safety measurement and monitoring in healthcare: a scoping review protocol

INTRODUCTION

Composite indicators of quality and safety in healthcare summarise performance across multiple indicators into a single performance measure. Composite indicators can identify domains and drivers of quality, improve the ability to detect differences, aid prioritisation for quality improvement and facilitate decision making about future healthcare needs. However, the use of composite indicators can be controversial, particularly when used to rank healthcare providers. Many of the concerns around transparency, appropriateness and uncertainty may be addressed by a robust and transparent development and review process.The aim of this scoping review is to describe methodologies used at each of the stages of development of composite indicators of quality and safety in healthcare. This review will provide those tasked with developing or reviewing composite indicators with a valuable consolidated analysis of a substantial and wide-ranging literature.

METHODS AND ANALYSIS

The framework proposed by the Joanna Briggs Institute and enhancements proposed by Peters et al (2015, 2017, 2020) will be used in conducting this scoping review, and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for scoping reviews will guide the reporting. Grey literature and peer-reviewed documents will be in-scope. Electronic databases (PubMed, Embase, CINAHL, ABI/INFORM and SafetyLit) will be searched, and publications will be screened by two reviewers. Discussion, policy and guidance publications will be included if they discuss any aspect of the methods used in the development of a composite indicator of quality or safety in a healthcare setting. The search period ranges from 1 January 2000 to 31 December 2022. Data extraction will capture information on 11 stages of composite indicator development, augmenting a 10-stage framework developed by the European Commission Joint Research Centre.

ETHICS AND DISSEMINATION

Ethical approval is not required. Review findings will be published in a peer-reviewed journal and presented at scientific conferences.

Quality in stroke care during the early phases of the COVID-19 pandemic: A nationwide study

Introduction:

Evidence-based early stroke care as reflected by fulfillment of process performance measures, is strongly related to better patient outcomes after stroke and transient ischemic attack (TIA). Detailed data on the resilience of stroke care services during the COVID-19 pandemic are limited. We aimed to examine the quality of early stroke care at Danish hospitals during the early phases of the COVID-19 pandemic.

Materials and methods:

We extracted data from Danish national health registries in five time periods (11 March, 2020–27 January, 2021) and compared these to a baseline pre-pandemic period (13 March, 2019–10 March, 2020). Quality of early stroke care was assessed as fulfilment of individual process performance measures and as a composite measure (opportunity-based score).

Results:

A total of 23,054 patients were admitted with stroke and 8153 with a TIA diagnosis in the entire period. On a national level, the opportunity-based score (95% confidence interval [CI]) at baseline for ischemic patients was 81.1% (80.8–81.4), for intracerebral hemorrhage (ICH) 85.5% (84.3–86.6), and for TIA 96.0% (95.3–96.1). An increase of 1.1% (0.1–2.2) and 1.5% (0.3–2.7) in the opportunity-based score was observed during the first national lockdown period for AIS and TIA followed by a decline of −1.3% (−2.2 to −0.4) in the gradual reopening phase for AIS indicators. We found a significant negative association between regional incidence rates and quality-of-care in ischemic stroke patients implying that quality decreases when admission rates increase.

Conclusion:

The quality of acute stroke/TIA care in Denmark remained high during the early phases of the pandemic and only minor fluctuations occurred.