Usage of primary and administrative data to measure the economic impact of quality improvement projects

Developing a tool to measure enactment of complex quality improvement interventions in healthcare

Quality improvement (QI) projects are common in healthcare settings and often involve interdisciplinary teams working together towards a common goal. Many interventions and programmes have been introduced through research to convey QI skills and knowledge to healthcare workers, however, a few studies have attempted to differentiate between what individuals 'learn' or 'know' versus their capacity to apply their learnings in complex healthcare settings. Understanding and differentiating between delivery, receipt, and enactment of QI skills and knowledge is important because while enactment alone does not guarantee desired QI outcomes, it might be reasonably assumed that 'better enactment' is likely to lead to better outcomes. This paper describes the development, application and validation of a tool to measure enactment of core QI skills and knowledge of a complex QI intervention in a healthcare setting. Based on the Institute for Healthcare Improvement's Model for Improvement, existing QI assessment tools, literature on enactment fidelity and our research protocols, 10 indicators related to core QI skills and knowledge were determined. Definitions and assessment criteria were tested and refined in five iterative cycles. Qualitative data from four QI teams in long-term care homes were used to test and validate the tool. The final measurement tool contains 10 QI indicators and a five-point scale. Inter-rater reliability ranged from good to excellent. Usability and acceptability among raters were considered high. This measurement tool assists in identifying strengths and weaknesses of a QI team and allows for targeted feedback on core QI components. The indicators developed in our tool and the approach to tool development may be useful in other health related contexts where similar data are collected.

How real can we get in generating real world evidence? Exploring the opportunities of routinely collected administrative data for evaluation of medical devices

Real-world data are considered a potentially valuable source of evidence for assessing medical technologies in clinical practice, but their widespread use is hampered by numerous challenges. Using the case of coronary stents in Italy, we investigate the potential of administrative databases for estimating costs and health outcomes associated with the use of medical devices in real world conditions. An administrative dataset was created ad hoc by merging hospital records from patients admitted between 2013 and 2019 for stent implantations with ambulatory records, pharmaceutical use data and vital statistics. Health outcomes were multifold: all-cause and cardiac mortality and myocardial infarction, within 30 days, 1, 2, 5 years. Costs were estimated from the National Health System perspective. We used multivariable Cox models and propensity score (PS) methods (PS matching; stratification on PS; inverse probability of treatment weighting using PS; PS adjustment). 257,907 coronary stents were implanted in 113,912 patients. For all health outcomes and follow-up times, and across all methods, patients receiving drug-eluting stents (DES) presented lower risk. For all-cause mortality, the DES patient advantage over bare-metal stent (BMS) patients declined over time but remained significant even at 5 years. For myocardial infarction, results remained quite stable. The DES group presented lower cumulative total costs (ranging from 3264 to 2363 Euros less depending on methods). Our results confirm the consolidated evidence of the benefits of DES compared to BMS. The consistency of results across methods suggests internal validity of the study, while highlighting strengths and limitations of each depending on research context. Administrative data yield great potential to perform comparative effectiveness and cost-effectiveness analysis of medical devices provided certain conditions are met.

The Hidden Pandemic: the Cost of Postoperative Complications

Purpose of Review

Population-based increases in ageing and medical co-morbidities are expected to substantially increase the incidence of expensive postoperative complications. This threatens the sustainability of essential surgical care, with negative impacts on patients' health and wellbeing.

Recent Findings

Identification of key high-risk areas, and implementation of proven cost-effective strategies to manage both outcome and cost across the end-to-end journey of the surgical episode of care, is clearly feasible. However, good programme design and formal cost-effectiveness analysis is critical to identify, and implement, true high value change.

Summary

Both outcome and cost need to be a high priority for both fundholders and clinicians in perioperative care, with the focus for both groups on delivering high-quality care, which in itself, is the key to good cost management.

Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review

OBJECTIVES

To profile the aims and characteristics of quality improvement (QI) initiatives conducted in Ireland, to review the quality of their reporting and to assess outcomes and costs.

DESIGN

Scoping review.

DATA SOURCES

Systematic searches were conducted in PubMed, Web of Science, Embase, Google Scholar, Lenus and rian.ie. Two researchers independently screened abstracts (n=379) and separately reviewed 43 studies identified for inclusion using a 70-item critique tool. The tool was based on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS), an appraisal instrument for QI intervention publications, and health economics reporting criteria. After reaching consensus, the final dataset was analysed using descriptive statistics. To support interpretations, findings were presented at a national stakeholder workshop.

ELIGIBILITY CRITERIA

QI studies implemented and evaluated in Ireland and published between January 2015 and April 2020.

RESULTS

The 43 studies represented various QI interventions. Most studies were peer-reviewed publications (n=37), conducted in hospitals (n=38). Studies mainly aimed to improve the 'effectiveness' (65%), 'efficiency' (53%), 'timeliness' (47%) and 'safety' (44%) of care. Fewer aimed to improve 'patient-centredness' (30%), 'value for money' (23%) or 'staff well-being' (9%). No study aimed to increase 'equity'. Seventy per cent of studies described 14 of 16 QI-MQCS dimensions. Least often studies reported the 'penetration/reach' of an initiative and only 35% reported health outcomes. While 53% of studies expressed awareness of costs, only eight provided at least one quantifiable figure for costs or savings. No studies assessed the cost-effectiveness of the QI.

CONCLUSION

Irish QI studies included in our review demonstrate varied aims and high reporting standards. Strategies are needed to support greater stimulation and dissemination of QI beyond the hospital sector and awareness of equity issues as QI work. Systematic measurement and reporting of costs and outcomes can be facilitated by integrating principles of health economics in QI education and guidelines.