More quality measures versus measuring what matters: a call for balance and parsimony

Measuring what matters: Context-specific indicators for assessing immunisation performance in Pacific Island Countries and Areas

Increasing countries' access to data can improve immunisation coverage through evidence-based decision-making. However, data collection and reporting is resource-intensive, so needs to be pragmatic, especially in low-and-middle-income countries. We aimed to identify which indicators are most important for measuring, and improving, national immunisation performance in Pacific Island Countries (PICs). We conducted an expert elicitation study, asking 13 experts involved in delivering immunisation programs, decision-makers, health information specialists, and global development partners across PICs to rate 41 indicators based on their knowledge of the feasibility and relevance of each indicator. We also asked experts their preferences for indicators to be retained or removed from a list of indicators for PICs. Experts participated in two rating rounds, with a discussion on the reasons for ratings before the second round. We calculated mean scores for feasibility and relevance, and ranked indicators based on experts' preferences and mean scores. We used framework analysis to identify reasons for selecting indicators. Experts agreed that certain indicators were essential to measure (e.g. data use in program planning and measles vaccination coverage), but preferences varied for most indicators. Preferences to include indicators in a set of indicators for PICs moderately correlated with scores for relevance (r = 0.68) and feasibility (r = 0.56). In discussions, experts highlighted usefulness for decision-making and ease of data collection, reporting and interpretation as the main reasons driving indicator selection. Country-specific factors such as health system factors, roles and influence of various immunisation actors, and macro-level factors (namely population size, distribution and mobility) affected relevance and feasibility, leading us to conclude that a single set of indicators for all PICs is inappropriate. Rather than having a strict set of indicators that all countries must measure and report against, performance indicators should be flexible, country-specific, and selected in consultation with immunisation actors who collect and use the data.

Clinical pharmacy key performance indicators for hospital inpatient setting: a systematic review

BACKGROUND

Key performance indicators (KPIs) are quantifiable measures used to monitor the quality of health services. Implementation guidelines for clinical pharmacy services (CPS) do not specify KPIs.

AIM

To assess the quality of the studies that have developed KPIs for CPS in inpatient hospital settings.

METHOD

A systematic review was conducted by searching in Web of Science, Scopus, and PubMed, supplemented with citation analyses and grey literature searches, to retrieve studies addressing the development of KPIs in CPS for hospital inpatients. Exclusions comprised drug- or disease-specific studies and those not written in English, French, Portuguese, or Spanish. The Appraisal of Indicators through Research and Evaluation (AIRE) instrument assessed methodological quality. Domain scores and an overall score were calculated using an equal-weight principle. KPIs were classified into structure, process, and outcome categories. The protocol is available at https://doi.org/10.17605/OSF.IO/KS2G3 .

RESULTS

We included thirteen studies that collectively developed 225 KPIs. Merely five studies scored over 50% on the AIRE instrument, with domains #3 (scientific evidence) and #4 (formulation and usage) displaying low scores. Among the KPIs, 8.4% were classified as structure, 85.8% as process, and 5.8% as outcome indicators. The overall methodological quality did not exhibit a clear association with a major focus on outcomes. None of the studies provided benchmarking reference values.

CONCLUSION

The KPIs formulated for evaluating CPS in hospital settings primarily comprised process measures, predominantly suggested by pharmacists, with inadequate evidence support, lacked piloting or validation, and consequently, were devoid of benchmarking reference values.

Venous thromboembolism performance measurement in the United States: An evolving landscape with many stakeholders

Venous thromboembolism (VTE), including deep vein thrombosis and pulmonary embolism, is a life-threatening, costly, and common preventable complication associated with hospitalization. Although VTE prevention strategies such as risk assessment and prophylaxis are available, they are not applied uniformly or systematically across US hospitals and healthcare systems. Hospital-level performance measurement has been used nationally to promote standardized approaches for VTE prevention and incentivize the adoption of guideline-based care management. Though most measures reflect care processes rather than outcomes, certain domains including diagnosis, treatment, and continuity of care remain unmeasured. In this article, we describe the development of VTE prevention measures from various stakeholders, measure strengths and limitations, publicly reported rates, the impact of technology and health policy on measure use, and perspectives on future options for surveillance and performance monitoring.

Performance measurement part I: Foundational knowledge for measure development

As medicine is moving toward performance and outcome-based payment and is transitioning away from productivity-based systems, value is now being appraised in healthcare through "performance measures." Over the past few decades, assessment of clinical performance in health care has been essential in ensuring safe and cost-effective patient care. The Centers for Medicare & Medicaid Services is further driving this change with measurable, outcomes-based national payer incentive payment systems. With the continually evolving requirements in health care reform focused on value-based care, there is a growing concern that clinicians, particularly dermatologists, may not understand the scientific rationale of health care quality measurement. As such, in order to help dermatologists understand the health care measurement science landscape to empower them to engage in the performance measure development and implementation process, the first article in this 2-part continuing medical education series reviews the value equation, historic and evolving policy issues, and the American Academy of Dermatology's approach to performance measurement development to provide the required foundational knowledge for performance measure developers.

Measurement that matters: A systematic review and modified Delphi of multidisciplinary colorectal cancer quality indicators

AIM

To develop a priority set of quality indicators (QIs) for use by colorectal cancer (CRC) multidisciplinary teams (MDTs).

METHODS

The review search strategy was executed in four databases from 2009-August 2019. Two reviewers screened abstracts/manuscripts. Candidate QIs and characteristics were extracted using a tailored abstraction tool and assessed for scientific soundness. To prioritize candidate indicators, a modified Delphi consensus process was conducted. Consensus was sought over two rounds; (1) multidisciplinary expert workshops to identify relevance to Australian CRC MDTs, and (2) an online survey to prioritize QIs by clinical importance.

RESULTS

A total of 93 unique QIs were extracted from 118 studies and categorized into domains of care within the CRC patient pathway. Approximately half the QIs involved more than one discipline (52.7%). One-third of QIs related to surgery of primary CRC (31.2%). QIs on supportive care (6%) and neoadjuvant therapy (6%) were limited. In the Delphi Round 1, workshop participants (n = 12) assessed 93 QIs and produced consensus on retaining 49 QIs including six new QIs. In Round 2, survey participants (n = 44) rated QIs and prioritized a final 26 QIs across all domains of care and disciplines with a concordance level > 80%. Participants represented all MDT disciplines, predominantly surgical (32%), radiation (23%) and medical (20%) oncology, and nursing (18%), across six Australian states, with an even spread of experience level.

CONCLUSION

This study identified a large number of existing CRC QIs and prioritized the most clinically relevant QIs for use by Australian MDTs to measure and monitor their performance.

Multiple case study of processes used by hospitals to select performance indicators: do they align with best practices?

Several health policy institutes recommend reducing the number of indicators monitored by hospitals to better focus on indicators most relevant to local contexts. To determine which indicators are the most appropriate to eliminate, one must understand how indicator selection processes are undertaken. This study classifies hospital indicator selection processes and analyzes how they align with practices outlined in the 5-P Indicator Selection Process Framework. This qualitative, multiple case study examined indicator selection processes used by four large acute care hospitals in Ontario, Canada. Data were collected through 13 semistructured interviews and document analysis. A thematic analysis compared processes to the 5-P Indicator Selection Process Framework. Two types of hospital indicator selection processes were identified. Hospitals deployed most elements found within the 5-P Indicator Selection Process Framework including setting clear aims, having governance structures, considering indicators required by health agencies, and categorizing indicators into strategic themes. Framework elements largely absent included: adopting evidence-based selection criteria; incorporating finance and human resources indicators; considering if indicators measure structures, processes, or outcomes; and engaging a broader set of end users in the selection process. Hospitals have difficulty in balancing how to monitor government-mandated indicators with indicators more relevant to local operations. Hospitals often do not involve frontline managers in indicator selection processes. Not engaging frontline managers in selecting indicators may risk hospitals only choosing government-mandated indicators that are not reflective of frontline operations or valued by those managers accountable for improving unit-level performance.

Quality of locally designed surveys in a quality improvement collaborative: review of survey validity and identification of common errors

OBJECTIVE

Surveys are a commonly used tool in quality improvement (QI) projects, but little is known about the standards to which they are designed and applied. We aimed to investigate the quality of surveys used within a QI collaborative, and to characterise the common errors made in survey design.

METHODS

Five reviewers (two research methodology and QI, three clinical and QI experts) independently assessed 20 surveys, comprising 250 survey items, that were developed in a North American cystic fibrosis lung transplant transition collaborative. Content Validity Index (CVI) scores were calculated for each survey. Reviewer consensus discussions decided an overall quality assessment for each survey and survey item (analysed using descriptive statistics) and explored the rationale for scoring (using qualitative thematic analysis).

RESULTS

3/20 surveys scored as high quality (CVI >80%). 19% (n=47) of survey items were recommended by the reviewers, with 35% (n=87) requiring improvements, and 46% (n=116) not recommended. Quality assessment criteria were agreed upon. Types of common errors identified included the ethics and appropriateness of questions and survey format; usefulness of survey items to inform learning or lead to action, and methodological issues with survey questions, survey response options; and overall survey design.

CONCLUSION

Survey development is a task that requires careful consideration, time and expertise. QI teams should consider whether a survey is the most appropriate form for capturing information during the improvement process. There is a need to educate and support QI teams to adhere to good practice and avoid common errors, thereby increasing the value of surveys for evaluation and QI. The methodology, quality assessment criteria and common errors described in this paper can provide a useful resource for this purpose.

Healthcare-associated infections and conditions in the era of digital measurement

As the third edition of the Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospitals is released with the latest recommendations for the prevention and management of healthcare-associated infections (HAIs), a new approach to reporting HAIs is just beginning to unfold. This next generation of HAI reporting will be fully electronic and based largely on existing data in electronic health record (EHR) systems and other electronic data sources. It will be a significant change in how hospitals report HAIs and how the Centers for Disease Control and Prevention (CDC) and other agencies receive this information. This paper outlines what that future electronic reporting system will look like and how it will impact HAI reporting.

Correlations among common emergency medicine physician performance measures: Mixed messages or balancing forces?

The increasing complexity of ED physician performance measures has resulted in significant challenges, including duplicative and conflicting measures that fail to account for different ED settings. We performed a cross sectional analysis of correlations between measures to characterize their relationships and determine if differences exist between academic versus non-academic ED settings. Pearson correlations were calculated for 12 measures among 220 ED physicians at 11 EDs. Higher admission rate was strongly correlated with higher CT utilization rate (R = 0.7, p < 0.01) and longer room to discharge time (R = 0.7, p < 0.01). Higher patients per hour was strongly correlated with shorter room to doctor time (R = -0.7, p < 0.01). Stronger measure correlations were found in the academic setting compared to the non-academic setting. Strong correlations between ED measures imply opportunities to reduce competing performance demands on clinicians. Differences in correlations at academic versus non-academic settings suggest that it may be inappropriate to apply the same performance standards across settings.

Time spent on documenting quality indicator data and associations between the perceived burden of documenting these data and joy in work among professionals in intensive care units in the Netherlands: a multicentre cross-sectional survey

OBJECTIVES

The number of indicators used to monitor and improve the quality of care is debatable and may influence professionals' joy in work. We aimed to assess intensive care unit (ICU) professionals' perceived burden of documenting quality indicator data and its association with joy in work.

DESIGN

Cross-sectional survey.

SETTING

ICUs of eight hospitals in the Netherlands.

PARTICIPANTS

Health professionals (ie, medical specialists, residents and nurses) working in the ICU.

MEASUREMENTS

The survey included reported time spent on documenting quality indicator data and validated measures for documentation burden (ie, such documentation being unreasonable and unnecessary) and elements of joy in work (ie, intrinsic and extrinsic motivation, autonomy, relatedness and competence). Multivariable regression analysis was performed for each element of joy in work as a separate outcome.

RESULTS

In total, 448 ICU professionals responded to the survey (65% response rate). The overall median time spent on documenting quality data per working day is 60 min (IQR 30-90). Nurses spend more time documenting these data than physicians (medians of 60 min vs 35 min, p<0.01). Most professionals (n=259, 66%) often perceive such documentation tasks as unnecessary and a minority (n=71, 18%) perceive them as unreasonable. No associations between documentation burden and measures of joy in work were found, except for the negative association between unnecessary documentations and sense of autonomy (β=-0.11, 95% CI -0.21 to -0.01, p=0.03).

CONCLUSIONS

Dutch ICU professionals spend substantial time on documenting quality indicator data they often regard as unnecessary. Despite the lacking necessity, documentation burden had limited impact on joy in work. Future research should focus on which aspects of work are affected by documentation burden and whether diminishing the burden improves joy in work.

National statutory reporting: not even ticking the boxes? The quality of 'Learning from Deaths' reporting in quality accounts within the NHS in England 2017-2020

INTRODUCTION

Regulation through statutory reporting is used in healthcare internationally to improve accountability, quality of care and patient safety. Since 2017, within the National Health Service (NHS) in England, NHS Secondary Care Trusts (NSCTs) are legally required to report annually both quantitative and qualitative information related to patient deaths within their care within their publicly available Quality Accounts as part of a countrywide patient safety programme: The Learning from Deaths (LfDs) programme.

METHOD

All LfDs reports published between 2017 (programme inception) and 2020 were reviewed and evaluated through a critical realist lens, quantitatively reported using descriptive statistics and qualitatively using reflexive thematic analysis.

RESULTS

In 2017/2018, 44% of NSCTs reported all six statutory elements of the LfDs reporting regulations, in 2019/2020 35% of NSCTs were reporting this information. A small number of NSCTs did not report any parts of the LfDs regulatory requirements between 2017 and 2020. Multiple qualitative themes arose from this study suggesting problematic engagement with the LfDs programme, erroneous reporting accuracy and errors in written communication.

CONCLUSIONS

The LfDs programme has, to some extent, reduced variation and improved consistency to the way that NSCTs identify, report and investigate deaths. However, 3 years into the LfDs programme, the majority of NSCTs are not reporting as required by law. This makes the validity of National statutory reporting in Quality Accounts within the NHS in England questionable as a regulatory process.

Patients' and kin's perspective on healthcare quality compared to Lachman's multidimensional quality model: Focus group interviews

OBJECTIVES

To identify key attributes of healthcare quality relevant to patients and kin and to compare them to Lachman's multidimensional quality model.

METHODS

Four focus groups with patients and kin were conducted using a semi-structured interview guide and a purposive sampling method. Classical content analysis and constant comparison method were used to focus data analysis on individual and group level.

RESULTS

Communication with patients, kin and professionals emerged as a new dimension from interview transcripts. Other identified key attributes largely corresponded with Lachman's multidimensional quality model. They were mainly classified in dimensions: 'Partnership and Co-Production', 'Dignity and Respect' and 'Effectiveness'. Technical quality dimensions were linked to organisational aspects of care in terms of staffing levels and time. The dimension 'Eco-friendly' was not addressed by patients or kin.

CONCLUSIONS

The results enhance the comprehension of healthcare quality and contribute to its academic understanding by validating Lachman's multidimensional quality model from patients' and kin's perspective. The model robustness is increased by including communication as a quality dimension surrounding technical domains and core values.

PRACTICE IMPLICATIONS

The key attributes can serve as a holistic framework for healthcare organisations to design their quality management system. An instrument can be developed to measure key attributes.

Which hospital-acquired conditions matter the most in trauma? An evidence-based approach for prioritizing trauma program improvement

BACKGROUND

Prevention of hospital-acquired conditions (HACs) is a focus of trauma center quality improvement. The relative contributions of various HACs to postinjury hospital outcomes are unclear. We sought to quantify and compare the impacts of six HACs on early clinical outcomes and resource utilization in hospitalized trauma patients.

METHODS

Adult patients from the 2013 to 2016 American College of Surgeons Trauma Quality Improvement Program Participant Use Data Files who required 5 days or longer of hospitalization and had an Injury Severity Score of 9 or greater were included. Multiple imputation with chained equations was used for observations with missing data. The frequencies of six HACs and five adverse outcomes were determined. Multivariable Poisson regression with log link and robust error variance was used to produce relative risk estimates, adjusting for patient-, hospital-, and injury-related factors. Risk-adjusted population attributable fractions estimates were derived for each HAC-outcome pair, with the adjusted population attributable fraction estimate for a given HAC-outcome pair representing the estimated percentage decrease in adverse outcome that would be expected if exposure to the HAC had been prevented.

RESULTS

A total of 529,856 patients requiring 5 days or longer of hospitalization were included. The incidences of HACs were as follows: pneumonia, 5.2%; urinary tract infection, 3.4%; venous thromboembolism, 3.3%; surgical site infection, 1.3%; pressure ulcer, 1.3%; and central line-associated blood stream infection, 0.2%. Pneumonia demonstrated the strongest association with in-hospital outcomes and resource utilization. Prevention of pneumonia in our cohort would have resulted in estimated reductions of the following: 22.1% for end organ dysfunction, 7.8% for mortality, 8.7% for prolonged hospitalization, 7.1% for prolonged intensive care unit stay, and 6.8% for need for mechanical ventilation. The impact of other HACs was comparatively small.

CONCLUSION

We describe a method for comparing the contributions of HACs to outcomes of hospitalized trauma patients. Our findings suggest that trauma program improvement efforts should prioritize pneumonia prevention.

LEVEL OF EVIDENCE

Therapeutic/Care Management; Level IV.

Putting measurement on a diet: development of a core set of indicators for quality improvement in the ICU using a Delphi method

Background

The number and efficacy of indicators used to monitor and improve the quality of care in Intensive Care Units (ICU) is debatable. This study aimed to select a consensus-based core set of indicators for effective quality improvement in the ICU.

Methods

A Delphi study with a panel of intensivists, ICU nurses, and former ICU patients or relatives (n = 34) from general, teaching, and academic hospitals. Panelists completed a questionnaire in which they scored 69 preselected quality indicators on relevance using a nine-point Likert scale. Indicators were categorized using the rated relevance score into: ‘accepted, ‘equivocal’ and ‘excluded’. Questionnaire results were discussed in focus groups to reach consensus on the final set.

Results

Response rates for the questionnaire and focus groups were 100 and 68%, respectively. Consensus was reached on a final set of 17 quality indicators including patient reported outcome measures (PROMs) and patient reported experience measures (PREMs). Other quality indicators relate to the organization and outcome of ICU care, including safety culture, ICU standardized mortality ratio, and the process indicator ‘learning from and improving after serious incidents’.

Conclusions

ICU clinicians and former patients and relatives developed a consensus-based core set of ICU quality indicators that is relatively short but comprehensive and particularly tailored to end-users needs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08236-3.

Selecting Performance Indicators and Targets in Health Care: An International Scoping Review and Standardized Process Framework

Objective

Health care organizations monitor hundreds of performance indicators. It is unclear what processes and criteria organizations use to identify the indicators they use, who is involved in these processes, how performance targets are set, and what the impacts of these processes are. The purpose of this study is to synthesize international approaches to indicator selection and develop a standardized process framework.

Methods

Using the PubMed and Web of Science search engines, a scoping review of peer reviewed and grey literature following PRISMA-ScR guidelines was conducted to identify documents describing indicator selection processes used by health systems. English-language papers from 11 countries published from 2010 to 2020 were included. Papers were thematically analyzed to develop a standardized process framework.

Results

The review included 33 peer-reviewed papers and 11 grey-literature documents. While there are common practices used in health care to select indicators, no single standardized process framework for indicator selection exists. Arbitrary or incomplete indicator selection processes risk over-measurement, lack of alignment with strategic and operational goals, lack of support by end-users, and paralyzed decision-making ability. By consolidating international practices, we developed the 5-P indicator selection process framework to mitigate process risks and support high-quality indicator selection processes.

Conclusion

The 5-P indicator selection process framework consists of five domains and 17 elements, and offers health care agencies a practical structure they can use to design indicator selection processes. The framework also provides researchers with a basis by which the implementation of these processes may be evaluated.

Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study

BACKGROUND

Quality indicators are registered to monitor and improve the quality of care. However, the number and effectiveness of quality indicators is under debate, and may influence the joy in work of physicians and nurses. Empirical data on the nature and consequences of the registration burden are lacking. The aim of this study was to identify and explore healthcare professionals' perceived burden due to quality registrations in hospitals, and the effect of this burden on their joy in work.

METHODS

A mixed methods observational study, including participative observations, a survey and semi-structured interviews in two academic hospitals and one teaching hospital in the Netherlands. Study participants were 371 healthcare professionals from an intensive care unit (ICU), a haematology department and others involved in the care of elderly patients and patients with prostate or gastrointestinal cancer.

RESULTS

On average, healthcare professionals spend 52.3 minutes per working day on quality registrations. The average number of quality measures per department is 91, with 1380 underlying variables. Overall, 57% are primarily registered for accountability purposes, 19% for institutional governance and 25% for quality improvement objectives. Only 36% were perceived as useful for improving quality in everyday practice. Eight types of registration burden were identified, such as an excessive number of quality registrations, and the lack of usefulness for improving quality and inefficiencies in the registration process. The time healthcare professionals spent on quality registrations was not correlated with any measure of joy in work. Perceived unreasonable registrations were negatively associated with healthcare professionals' joy in work (intrinsic motivation and autonomy). Healthcare professionals experienced quality registrations as diverting time from patient care and from actually improving quality.

CONCLUSION

Registering fewer quality indicators, but more of what really matters to healthcare professionals, is key to increasing the effectiveness of registrations for quality improvement and governance. Also the efficiency of quality registrations should be increased through staffing and information and communications technology solutions to reduce the registration burden experienced by nurses and physicians.

Development of an international standard set of patient-centred outcome measures for overall paediatric health: a consensus process

OBJECTIVE

To develop an Overall Pediatric Health Standard Set (OPH-SS) of outcome measures that captures what matters to young people and their families and recognising the biopsychosocial aspects of health for all children and adolescents regardless of health condition.

DESIGN

A modified Delphi process.

SETTING

The International Consortium for Health Outcomes Measurement convened an international Working Group (WG) comprised of 23 international experts from 12 countries in the field of paediatrics, family medicine, psychometrics as well as patient advisors. The WG participated in 11 video-conferences, through a modified Delphi process and 9 surveys between March 2018 and January 2020 consensus was reached on a final recommended health outcome standard set. By a literature review conducted in March 2018, 1136 articles were screened for clinician and patient-reported or proxy-reported outcomes. Further, 4315 clinical trials and 12 paediatric health surveys were scanned. Between November 2019 and January 2020, the final standard set was endorsed by a patient validation (n=270) and a health professional (n=51) survey.

RESULTS

From a total of 63 identified outcomes, consensus was formed on a standard set of outcome measures that comprises 10 patient-reported outcomes, 5 clinician-reported measures, and 6 case-mix variables. The four developmental age-specific packages (ie, 0-5, 6-12, 13-17, 18-24 years) include either five or six measures with an average time for completion of 20 min.

CONCLUSIONS

The OPH-SS is a starting point to drive value-based paediatric healthcare delivery from a global perspective for enhancing child and adolescent physical health and psychosocial well-being.

Evaluation of Clinical and Economic Outcomes Following Implementation of a Medicare Pay-for-Performance Program for Surgical Procedures

IMPORTANCE

Surgical complications increase hospital costs by approximately $20 000 per admission and extend hospital stays by 9.7 days. Improving surgical care quality and reducing costs is needed for patients undergoing surgery, health care professionals, hospitals, and payers.

OBJECTIVE

To evaluate the association of the Hospital-Acquired Conditions Present on Admission (HAC-POA) program, a mandated national pay-for-performance program by the Centers for Medicare & Medicaid Services, with surgical care quality and costs.

DESIGN, SETTING, AND PARTICIPANTS

A cross-sectional study of Medicare inpatient surgical care stays from October 2004 through September 2017 in the US was conducted. The National Inpatient Sample and a propensity score-weighted difference-in-differences analysis of hospital stays with associated primary surgical procedures was used to compare changes in outcomes for the intervention and control procedures before and after HAC-POA program implementation. The sample consisted of 1 317 262 inpatient surgical episodes representing 1 198 665 stays for targeted procedures and 118 597 stays for nontargeted procedures. Analyses were performed between November 1, 2020, and May 7, 2021.

EXPOSURES

Implementation of the HAC-POA program for the intervention procedures included in this study (fiscal year 2009).

MAIN OUTCOMES AND MEASURES

Incidence of surgical site infections and deep vein thrombosis, length of stay, in-hospital mortality, and hospital costs. Analyses were adjusted for patient and hospital characteristics and indicators for procedure type, hospital, and year.

RESULTS

In our propensity score-weighted sample, the intervention procedures group comprised 1 047 351 (88.5%) individuals who were White and 742 734 (60.6%) women; mean (SD) age was 75 (6.9) years. The control procedures group included 94 715 (88.0%) individuals who were White, and 65 436 (60.6%) women; mean (SD) age was 75 (7.1) years. After HAC-POA implementation, the incidence of surgical site infections in targeted procedures decreased by 0.3 percentage points (95% CI, -0.5 to -0.1 percentage points; P = .02) compared with nontargeted procedures. The program was associated with a reduction in length of stay by 0.5 days (95% CI, -0.6 to -0.4 days; P < .001) and hospital costs by 8.1% (95% CI, -10.2% to -6.1%; P < .001). No significant changes in deep vein thrombosis incidence and mortality were noted.

CONCLUSIONS AND RELEVANCE

The findings of this study suggest that the HAC-POA program is associated with small decreases in surgical site infection and length of stay and moderate decreases in hospital costs for patients enrolled in Medicare. Policy makers may consider these findings when evaluating the continuation and expansion of this program for other surgical procedures, and payers may want to consider adopting a similar policy.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

Stakeholder Perceptions of Key Aspects of High-Quality Cancer Care to Assess with Patient Reported Outcome Measures: A Systematic Review

Simple Summary

We conducted a review to identify important symptoms reported by patients on questionnaires (e.g., pain) that can be used to compare cancer centers on how well they provide care. For example, cancer centers could be compared on the percentage of patients with controlled pain after adjusting for demographic and clinical characteristics. Standard review methods were used to identify studies through August 2020. Searches generated 1813 articles and 1779 were coded as not relevant. The remaining 34 studies showed that patients, caregivers, clinicians, and healthcare administrators identify psychosocial care (e.g., distress) and symptom management as critical parts of high-quality care. Patients and caregivers also perceive that maintaining physical function and daily activities are important. Clinicians and healthcare administrators perceive control of specific symptoms to be important (e.g., pain, poor sleep, diarrhea). Results were used to inform testing of symptom questionnaires to compare the quality of care provided by six cancer centers.

Abstract

Performance measurement is the process of collecting, analyzing, and reporting standardized measures of clinical performance that can be compared across practices to evaluate how well care was provided. We conducted a systematic review to identify stakeholder perceptions of key symptoms and health domains to test as patient-reported performance measures in oncology. Stakeholders included cancer patients, caregivers, clinicians, and healthcare administrators. Standard review methodology was used, consistent with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). MEDLINE/PubMed, EMBASE, and the Cochrane Library were searched to identify relevant studies through August 2020. Four coders independently reviewed entries and conflicts were resolved by a fifth coder. Efficacy and effectiveness studies, and studies focused exclusively on patient experiences of care (e.g., communication skills of providers) were excluded. Searches generated 1813 articles and 1779 were coded as not relevant, leaving 34 international articles for extraction. Patients, caregivers, clinicians, and healthcare administrators prioritize psychosocial care (e.g., distress) and symptom management for patient-reported performance measures. Patients and caregivers also perceive that maintaining physical function and daily activities are critical. Clinicians and administrators perceive control of specific symptoms to be critical (gastrointestinal symptoms, pain, poor sleep). Results were used to inform testing at six US cancer centers.

How quality improvement collaboratives work to improve healthcare in care homes: a realist evaluation

BACKGROUND

Quality improvement collaboratives (QICs) bring together multidisciplinary teams in a structured process to improve care quality. How QICs can be used to support healthcare improvement in care homes is not fully understood.

METHODS

A realist evaluation to develop and test a programme theory of how QICs work to improve healthcare in care homes. A multiple case study design considered implementation across 4 sites and 29 care homes. Observations, interviews and focus groups captured contexts and mechanisms operating within QICs. Data analysis classified emerging themes using context-mechanism-outcome configurations to explain how NHS and care home staff work together to design and implement improvement.

RESULTS

QICs will be able to implement and iterate improvements in care homes where they have a broad and easily understandable remit; recruit staff with established partnership working between the NHS and care homes; use strategies to build relationships and minimise hierarchy; protect and pay for staff time; enable staff to implement improvements aligned with existing work; help members develop plans in manageable chunks through QI coaching; encourage QIC members to recruit multidisciplinary support through existing networks; facilitate meetings in care homes and use shared learning events to build multidisciplinary interventions stepwise. Teams did not use measurement for change, citing difficulties integrating this into pre-existing and QI-related workload.

CONCLUSIONS

These findings outline what needs to be in place for health and social care staff to work together to effect change. Further research needs to consider ways to work alongside staff to incorporate measurement for change into QI.

A mixed-methods study of challenges experienced by clinical teams in measuring improvement

OBJECTIVE

Measurement is an indispensable element of most quality improvement (QI) projects, but it is undertaken to variable standards. We aimed to characterise challenges faced by clinical teams in undertaking measurement in the context of a safety QI programme that encouraged local selection of measures.

METHODS

Drawing on an independent evaluation of a multisite improvement programme (Safer Clinical Systems), we combined a qualitative study of participating teams' experiences and perceptions of measurement with expert review of measurement plans and analysis of data collected for the programme. Multidisciplinary teams of frontline clinicians at nine UK NHS sites took part across the two phases of the programme between 2011 and 2016.

RESULTS

Developing and implementing a measurement plan against which to assess their improvement goals was an arduous task for participating sites. The operational definitions of the measures that they selected were often imprecise or missed important details. Some measures used by the teams were not logically linked to the improvement actions they implemented. Regardless of the specific type of data used (routinely collected or selected ex novo), the burdensome nature of data collection was underestimated. Problems also emerged in identifying and using suitable analytical approaches.

CONCLUSION

Measurement is a highly technical task requiring a degree of expertise. Simply leveraging individual clinicians' motivation is unlikely to defeat the persistent difficulties experienced by clinical teams when attempting to measure their improvement efforts. We suggest that more structural initiatives and broader capability-building programmes should be pursued by the professional community. Improving access to, and ability to use repositories of validated measures, and increasing transparency in reporting measurement attempts, is likely to be helpful.

Content comparison and person-centeredness of standards for quality improvement in cardiovascular health care

Background

Quality standards are important for improving health care by providing compelling evidence for best practice. High quality person-centered health care requires information on patients' experience of disease and of functioning in daily life.

Objective

To analyze and compare the content of five Swedish National Quality Registries (NQRs) and two standard sets of the International Consortium of Health Outcomes Measurement (ICHOM) related to cardiovascular diseases.

Materials and methods

An analysis of 2588 variables (= data items) of five NQRs—the Swedish Registry of Congenital Heart Disease, Swedish Cardiac Arrest Registry, Swedish Catheter Ablation Registry, Swedish Heart Failure Registry, SWEDEHEART (including four sub-registries) and two ICHOM standard sets–the Heart Failure Standard Set and the Coronary Artery Disease Standard Set. According to the name and definition of each variable, the variables were mapped to Donabedian’s quality criteria, whereby identifying whether they capture health care processes or structures or patients’ health outcomes. Health outcomes were further analyzed whether they were clinician- or patient-reported and whether they capture patients’ physiological functions, anatomical structures or activities and participation.

Results

In total, 606 variables addressed process quality criteria (31%), 58 structure quality criteria (3%) and 760 outcome quality criteria (38%). Of the outcomes reported, 85% were reported by clinicians and 15% by patients. Outcome variables addressed mainly ‘Body functions’ (n = 392, 55%) or diseases (n = 209, 29%). Two percent of all documented data captured patients’ lived experience of disease and their daily activities and participation (n = 51, 3% of all variables).

Conclusions

Quality standards in the cardiovascular field focus predominately on processes (e.g. treatment) and on body functions-related outcomes. Less attention is given to patients’ lived experience of disease and their daily activities and participation. The results can serve as a starting-point for harmonizing data and developing a common person-centered quality indicator set.

Improving quality of life assessments for high-need adult Medicaid service users with mental health conditions

PURPOSE

There is a lack of consensus on how to evaluate health and social service programs for people with mental health (MH) conditions. Having service users be the primary decision makers in selecting outcome measures can inform a meaningful evaluation strategy. We sought to identify the quality of life (QoL) survey preferences of high-need adult service users with MH conditions.

METHODS

A systematic review identified generic, self-reported QoL surveys with evidence of validity in MH populations of interest. An advisory panel selected the most promising surveys to assess the success of programs like Medicaid for MH service users. Three groups of high-need, adult service users with MH conditions and one group of direct care staff ranked the surveys from the advisory panel, and generated and ranked characteristics that were desirable or undesirable in a QoL survey.

RESULTS

Twenty-two surveys met the inclusion criteria. Of the six surveys selected by the advisory panel, groups of service users and direct care staff most preferred the Warwick-Edinburg Mental Well-being Scale (WEMWBS). The WEMWBS best embodied the features prioritized by the groups: to have a user-friendly format and positive focus, to be clearly worded and brief, and to avoid presumptive or unrealistic items. Service user groups appreciated survey topics most amenable to self-report, such as satisfaction with relationships.

CONCLUSION

Using QoL surveys that service users prefer can reduce the chance that deteriorating QoL is going unchecked, and increase the chance that decisions based on survey findings are meaningful to service users.

Measures of Care Coordination at Inpatient Psychiatric Facilities and the Medicare 30-Day All-Cause Readmission Rate

OBJECTIVE

Since late 2012, the Medicare Inpatient Psychiatric Facility Quality Reporting (IPFQR) program of the Centers for Medicare and Medicaid Services (CMS) has required inpatient psychiatric facilities to collect and publicly report a suite of quality measures. This study explored the association between facility-level 30-day risk-adjusted all-cause readmission (medical or psychiatric) after psychiatric hospitalization (READM-30-IPF) and care coordination process measures in the IPFQR program.

METHODS

The study used publicly reported IPFQR facility-level performance data of the Hospital Compare Web site for 1,343 inpatient psychiatric facilities, reflecting performance from July 2015 to June 2017. The authors used a cross-sectional design and linear regression models controlling for hospital and community characteristics and using state as fixed effect.

RESULTS

The mean±SD facility-level READM-30-IPF was 20%±3%, with substantial variation by facility type, ownership status, rurality, and percentage of racial-ethnic minority residents in the county. Regression results showed that facilities with performance in the top tercile on the measure of 7-day mental health follow-up after discharge had readmission rates significantly lower than facilities in the bottom tercile (coefficient=-0.58, p<0.01), although the magnitude of this difference was small. READM-30-IPF, however, did not vary by facilities' performance on measures of discharge plan creation and transmission.

CONCLUSIONS

Results suggest that facilities have substantial opportunities to reduce readmissions after psychiatric hospitalization. The association between hospital performance on care coordination process measures and the all-cause readmission measure currently included in the IPFQR program was minimal. The CMS should evaluate whether the IPFQR measures adequately capture compliance with evidence-based processes and desired outcomes.

Examining the Relationship of an All-Cause Harm Patient Safety Measure and Critical Performance Measures at the Frontline of Care

Background

In 2015, the Institute of Medicine Vital Signs report called for a new patient safety composite measure to lessen the reporting burden of patient harm. Before this report, two patient safety organizations had developed an electronic all-cause harm measurement system leveraging data from the electronic health record, which identified and grouped harms into five broad categories and consolidated them into one all-cause harm outcome measure.

Objectives

The objective of this study was to examine the relationship between this all-cause harm patient safety measure and the following three performance measures important to overall hospital safety performance: safety culture, employee engagement, and patient experience.

Methods

We studied the relationship between all-cause harm and three performance measures on eight inpatient care units at one hospital for 7 months.

Results

The findings demonstrated strong correlations between an all-cause harm measure and patient safety culture, employee engagement, and patient experience at the hospital unit level. Four safety culture domains showed significant negative correlations with all-cause harm at a P value of 0.05 or less. Six employee engagement domains were significantly negatively correlated with all-cause harm at a P value of 0.01 or less, and six of the ten patient experience measures were significantly correlated with all-cause harm at a P value of 0.05 or less.

Conclusions

The results show that there is a strong relationship between all-cause harm and these performance measures indicating that when there is a positive patient safety culture, a more engaged employee, and a more satisfying patient experience, there may be less all-cause harm.

Evaluating the Impact of Oncology Care Model Reporting Requirements on Biomarker Testing and Treatment

PURPOSE

The Oncology Care Model (OCM) is Medicare's first alternative payment model program for patients with cancer. As of October 2017, participating practices were required to report biomarker testing of patients with advanced non-small-cell lung cancer (aNSCLC). Our objective was to evaluate the effect of this OCM reporting requirement on quality of care.

METHODS

We selected patients with aNSCLC receiving care in practices in a nationwide de-identified electronic health record-derived database. We used an adjusted difference-in-differences (DID) logistic regression model to compare changes in biomarker testing rates (EGFR, ROS1, and ALK) and receipt of biomarker-guided therapy between patients in OCM versus non-OCM practices, before and after OCM implementation.

RESULTS

The analysis included 14,048 patients from 45 OCM practices (n = 8,151) and 105 non-OCM practices (n = 5,897). The overall unadjusted rates for biomarker testing and receipt of biomarker-guided therapy increased over the study period (2011-2018) in both OCM (55.5% v 71.6%; 89.8% v 94.6%, respectively) and non-OCM (55.2% v 69.7%; 90.1% v 95.2%, respectively) practices. In the adjusted DID model, the rates of biomarker testing (odds ratio [OR], 1.09 [95% CI, 0.88 to 1.34]; P = .45) and receipt of biomarker-guided therapy (OR, 0.87 [95% CI, 0.52 to 1.45]; P = .58) were similar between OCM and non-OCM practices.

CONCLUSION

OCM biomarker documentation and reporting requirements did not appear to increase the proportions of patients with aNSCLC who underwent testing or who received biomarker-guided therapy in OCM versus non-OCM practices.

Steering by their own lights: Why regulators across Europe use different indicators to measure healthcare quality

•

Indicator sets differ in how they define, measure, and assess healthcare quality.

•

National sets shaped by varying governance traditions and healthcare system configuration.

•

Targeting of quality dimensions and hospital activities shaped by system-specific ‘demand-side’ pressures.

•

Measurement styles shaped by ‘supply-side’ constraints on data access and indicator construction.

•

International benchmarking is easier when healthcare systems and governance traditions are similar.

Despite widespread faith that quality indicators are key to healthcare improvement and regulation, surprisingly little is known about what is actually measured in different countries, nor how, nor why. To address that gap, this article compares the official indicator sets--comprising some 1100 quality measures-- used by statutory hospital regulators in England, Germany, France, and the Netherlands. The findings demonstrate that those countries’ regulators strike very different balances in: the dimensions of quality they assess (e.g. between safety, effectiveness, and patient-centredness); the hospital activities they target (e.g. between clinical and non-clinical activities and management); and the ‘Donabedian’ measurement style of their indicators (between structure, process and outcome indicators). We argue that these contrasts reflect: i) how the distinctive problems facing each country’s healthcare system create different ‘demand-side’ pressures on what national indicator sets measure; and ii) how the configuration of national healthcare systems and governance traditions create ‘supply-side’ constraints on the kinds of data that regulators can use for indicator construction. Our analysis suggests fundamental differences in the meaning of quality and its measurement across countries that are likely to impede international efforts to benchmark quality and identify best practice.

Measures to Improve Diagnostic Safety in Clinical Practice

Timely and accurate diagnosis is foundational to good clinical practice and an essential first step to achieving optimal patient outcomes. However, a recent Institute of Medicine report concluded that most of us will experience at least one diagnostic error in our lifetime. The report argues for efforts to improve the reliability of the diagnostic process through better measurement of diagnostic performance. The diagnostic process is a dynamic team-based activity that involves uncertainty, plays out over time, and requires effective communication and collaboration among multiple clinicians, diagnostic services, and the patient. Thus, it poses special challenges for measurement. In this paper, we discuss how the need to develop measures to improve diagnostic performance could move forward at a time when the scientific foundation needed to inform measurement is still evolving. We highlight challenges and opportunities for developing potential measures of "diagnostic safety" related to clinical diagnostic errors and associated preventable diagnostic harm. In doing so, we propose a starter set of measurement concepts for initial consideration that seem reasonably related to diagnostic safety and call for these to be studied and further refined. This would enable safe diagnosis to become an organizational priority and facilitate quality improvement. Health-care systems should consider measurement and evaluation of diagnostic performance as essential to timely and accurate diagnosis and to the reduction of preventable diagnostic harm.

Primary Care Practices' Ability to Report Electronic Clinical Quality Measures in the EvidenceNOW Southwest Initiative to Improve Heart Health

IMPORTANCE

The capability and capacity of primary care practices to report electronic clinical quality measures (eCQMs) are questionable.

OBJECTIVE

To determine how quickly primary care practices can report eCQMs and the practice characteristics associated with faster reporting.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study examined an initiative (EvidenceNOW Southwest) to enhance primary care practices' ability to adopt evidence-based cardiovascular care approaches: aspirin prescribing, blood pressure control, cholesterol management, and smoking cessation (ABCS). A total of 211 primary care practices in Colorado and New Mexico participating in EvidenceNOW Southwest between February 2015 and December 2017 were included.

INTERVENTIONS

Practices were instructed on eCQM specifications that could be produced by an electronic health record, a registry, or a third-party platform. Practices received 9 months of support from a practice facilitator, a clinical health information technology advisor, and the research team. Practices were instructed to report their baseline ABCS eCQMs as soon as possible.

MAIN OUTCOMES AND MEASURES

The main outcome was time to report the ABCS eCQMs. Cox proportional hazards models were used to examine practice characteristics associated with time to reporting.

RESULTS

Practices were predominantly clinician owned (48%) and in urban or suburban areas (71%). Practices required a median (interquartile range) of 8.2 (4.6-11.9) months to report any ABCS eCQM. Time to report differed by eCQM: practices reported blood pressure management the fastest (median [interquartile range], 7.8 [3.5-10.4] months) and cholesterol management the slowest (median [interquartile range], 10.5 [6.6 to >12] months) (log-rank P < .001). In multivariable models, the blood pressure eCQM was reported more quickly by practices that participated in accountable care organizations (hazard ratio [HR], 1.88; 95% CI, 1.40-2.53; P < .001) or participated in a quality demonstration program (HR, 1.58; 95% CI, 1.14-2.18; P = .006). The cholesterol eCQM was reported more quickly by practices that used clinical guidelines for cardiovascular disease management (HR, 1.35; 95% CI, 1.18-1.53; P < .001). Compared with Federally Qualified Health Centers, hospital-owned practices had greater ability to report blood pressure eCQMs (HR, 2.66; 95% CI, 95% CI, 1.73-4.09; P < .001), and clinician-owned practices had less ability to report cholesterol eCQMs (HR, 0.52; 95% CI, 0.35-0.76; P < .001).

CONCLUSIONS AND RELEVANCE

In this study, time to report eCQMs varied by measure and practice type, with very few practices reporting quickly. Practices took longer to report a new cholesterol measure than other measures. Programs that require eCQM reporting should consider the time and effort practices must exert to produce reports. Practices may benefit from additional support to succeed in new programs that require eCQM reporting.

From Board to Bedside: How the Application of Financial Structures to Safety and Quality Can Drive Accountability in a Large Health Care System

BACKGROUND

As the health care system in the United States places greater emphasis on the public reporting of quality and safety data and its use to determine payment, provider organizations must implement structures that ensure discipline and rigor regarding these data. An academic health system, as part of a performance management system, applied four key components of a financial reporting structure to support the goal of top-to-bottom accountability for improving quality and safety.

FOUR KEY COMPONENTS OF A FINANCIAL REPORTING STRUCTURE

The four components implemented by Johns Hopkins Medicine were governance, accountability, reporting of consolidated quality performance statements, and auditing. Governance is provided by the health system's Patient Safety and Quality Board Committee, which reviews goals and strategy for patient safety and quality, reviews quarterly performance for each entity, and holds organizational leaders accountable for performance. An accountability plan includes escalating levels of review corresponding to the number of months an entity misses the defined performance target for a measure. A consolidated quality statement helps inform the Patient Safety and Quality Board Committee and leadership on key quality and safety issues. An audit evaluates the efficiency and effectiveness of processes for data collection, validation, and storage, as to ensure the accuracy and completeness of quality measure reporting.

CONCLUSION

If hospitals and health systems truly want to prioritize improvements in safety and quality, they will need to create a performance management system that ensures data validity and supports performance accountability. Without valid data, it is difficult to know whether a performance gap is due to data quality or clinical quality.

A Quality Strategy to Advance the Triple Aim in California's Medicaid Program

The California Department of Health Care Services (DHCS) administers the nation's largest Medicaid program. In 2012, DHCS developed a Quality Strategy modeled after the National Quality Strategy to guide the Department's activities aimed at advancing the Triple Aim. The Triple Aim seeks to improve the patient experience of care and the health of populations as well as reduce the per capita cost of health care. An academic team was contracted to assist DHCS in developing the strategy, which also was informed by extensive stakeholder input, an advisory committee, and a comprehensive inventory of DHCS quality improvement (QI) activities. From 2012 to 2018, the strategy included 129 unique QI activities. Most activities were intended to deliver more effective, efficient, affordable care or to advance disease prevention. This qualitative assessment of the DHCS Quality Strategy provides insights that may inform other Medicaid programs or large health systems as they develop quality strategies.

Quality indicators and their regular use in clinical practice: results from a survey among users of two cardiovascular National Registries in Sweden

Objective

To examine the regular use of quality indicators from Swedish cardiovascular National Quality Registries (NQRs) by clinical staff; particularly differences in use between the two NQRs and between nurses and physicians.

Design

Cross-sectional online survey study.

Setting

Two Swedish cardiovascular NQRs: (a) Swedish Heart Failure Registry and (b) Swedeheart.

Participants

Clinicians (n =185; 70% nurses, 26% physicians) via the NQRs' email networks.

Main Outcome Measures

Frequency of NQR use for (a) producing healthcare activity statistics; (b) comparing results between similar departments; (c) sharing results with colleagues; (d) identifying areas for quality improvement (QI); (e) surveilling the impact of QI efforts; (f) monitoring effects of implementation of new treatment methods; (g) doing research and (h) educating and informing healthcare professionals and patients.

Results

Median use of NQRs was 10 times a year (25th and 75th percentiles range: 3-23 times/year). Quality indicators from the NQRs were used mainly for producing healthcare activity statistics. Median use of Swedeheart was six times greater than Swedish Heart Failure Registry (SwedeHF; P < 0.000). Physicians used the NQRs more than twice as often as nurses (18 vs. 7.5 times/year; P < 0.000) and perceived NQR work more often as meaningful. Around twice as many Swedeheart users had the role to participate in data analysis and in QI efforts compared to SwedeHF users.

Conclusions

Most respondents used quality indicators from the two cardiovascular NQRs infrequently (<3 times/year). The results indicate that linking registration of quality indicators to using them for QI activities increases their routine use and makes them meaningful tools for professionals.

Days at Home after Surgery: An Integrated and Efficient Outcome Measure for Clinical Trials and Quality Assurance

BACKGROUND

Surgical audit, sometimes including public reporting, is an important foundation of high quality health care. We aimed to assess the validity of a novel outcome metric, days at home up to 30 days after surgery, as a surgical outcome measure in clinical trials and quality assurance.

METHODS

This was a multicentre, registry-based cohort study. We used prospectively collected hospital and national healthcare registry data obtained from patients aged 18 years or older undergoing a broad range of surgeries in Sweden over a 10-year period. The association between days at home up to 30 days after surgery and patient (older age, poorer physical status, comorbidity) and surgical (elective or non-elective, complexity, duration) risk factors, process of care outcomes (re-admissions, discharge destination), clinical outcomes (major complications, 30-day mortality) and death up to 1 year after surgery were measured.

FINDINGS

From January, 2005, to December, 2014, we obtained demographic and perioperative data on 636,885 patients from 21 Swedish hospitals. Mortality at 30 days and one year was 1.8% and 7.3%, respectively. The median (IQR) days at home up to 30 days after surgery was 27 (23-29), being significantly lower among high-risk patients, those recovering from more complex surgical procedures, and suffering serious postoperative complications (all p < 0.0001). Patients with 8 days or less at home up to 30 days after surgery had a nearly 7-fold higher risk of death up to 1 year postoperatively when compared with those with 29 or 30 days at home (adjusted HR 6.78 [95% CI: 6.44-7.13]).

INTERPRETATION

Days at home up to 30 days after surgery is a valid, easy to measure patient-centred outcome metric. It is highly sensitive to changes in surgical risk and impact of complications, and has prognostic importance; it is therefore a valuable endpoint for perioperative clinical trials and quality assurance.

FUNDING

Swedish National Research Council Medicine and Stockholm County Council ALF-project grant (LE), and the Australian National Health and Medical Research Council (PM).

Unintended consequences and challenges of quality measurements in dentistry

BACKGROUND

In recent years, several state dental programs, researchers and the Dental Quality Alliance (DQA) have sought to develop baseline quality measures for dentistry as a way to improve health outcomes, reduce costs and enhance patient experiences. Some of these measures have been tested and validated for various population groups. However, there are some unintended consequences and challenges with quality measurement in dentistry as observed from our previous work on refining and transforming dental quality measures into e-measures.

MAIN BODY

Some examples of the unintended consequences and challenges associated with implementing dental quality measures include: a de-emphasis on patient-centeredness with process-based quality measures, an incentivization of unethical behavior due to fee-for-service reimbursement systems, the risk of compromising patient and provider autonomy with plan-level measures, a disproportionate benefits of dental quality measurement going toward payers, and the risk of alienating smaller dental offices due to the resource-intensive nature of quality measurement.

CONCLUSION

As our medical counterparts have embraced quality measurement for improved health outcomes, so too must the dental profession. Our ultimate goal is to ensure the delivery of high quality, patient-centered dental care and effective quality measurement is the first step. By continuously monitoring the performance of dental quality measures and their continued refinement when unintended consequences are observed, we can improve patient and population health outcomes.

Role of Nursing Informatics in the Automation of Pneumonia Quality Measure Data Elements

Core measures are standard metrics to reflect the processes of care provided by hospitals. Hospitals in the United States are expected to extract data from electronic health records, automated computation of core measures, and electronic submission of the quality measures data. Traditional manual calculation processes are time intensive and susceptible to error. Automated calculation has the potential to provide timely, accurate information, which could guide quality-of-care decisions, but this vision has yet to be achieved. In this study, nursing informaticists and data analysts implemented a method to automatically extract data elements from electronic health records to calculate a core measure. We analyzed the sensitivity, specificity, and accuracy of core measure data elements extracted via SQL query and compared the results to manually extracted data elements. This method achieved excellent performance for the structured data elements but was less efficient for semistructured and unstructured elements. We analyzed challenges in automating the calculation of quality measures and proposed a rule-based (hybrid) approach for semistructured and unstructured data elements.

A National Strategy For Palliative Care

In 2014 the World Health Organization called for palliative care to be integrated as an essential element of the health care continuum. Yet in 2017 US palliative care services are found largely in hospitals, and hospice care, which is delivered primarily in the home, is limited to people who are dying soon. The majority of Americans with a serious illness are not dying; are living at home, in assisted living facilities, or in nursing homes; and have limited access to palliative care. Most health care providers lack knowledge about and skills in pain and symptom management, communication, and care coordination, and both the public and health professionals are only vaguely aware of the benefits of palliative care and how and when to access it. The lack of policy supports for palliative care contributes to preventable suffering and low-value care. In this article we outline the need for a national palliative care strategy to ensure reliable access to high-quality palliative care for Americans with serious medical illnesses. We review approaches employed by other countries, list the participants needed to develop and implement an actionable strategy, and identify analogous US national health initiatives to inform a process for implementing the strategy.

Designing for Clinical Change: Creating an Intervention to Implement New Statin Guidelines in a Primary Care Clinic

Background

Recent clinical practice guidelines from major national organizations, including a joint United States Department of Veterans Affairs (VA) and Department of Defense (DoD) committee, have substantially changed recommendations for the use of the cholesterol-lowering statin medications after years of relative stability. Because statin medications are among the most commonly prescribed treatments in the United States, any change in their use may have significant implications for patients and providers alike. Prior research has shown that effective implementation interventions should be both user centered and specifically chosen to address identified barriers.

Objective

The objectives of this study were to identify potential determinants of provider uptake of the new statin guidelines and to use that information to tailor a coordinated and streamlined local quality improvement intervention focused on prescribing appropriate statins.

Methods

We employed user-centered design principles to guide the development and testing of a multicomponent guideline implementation intervention to improve statin prescribing. This paper describes the intervention development process whereby semistructured qualitative interviews with providers were conducted to (1) illuminate the knowledge, attitudes, and behaviors of providers and (2) elicit feedback on intervention prototypes developed to align with and support the use of the VA/DoD guidelines. Our aim was to use this information to design a local quality improvement intervention focused on statin prescribing that was tailored to the needs of primary care providers at our facility. Cabana’s Clinical Practice Guidelines Framework for Improvement and Nielsen’s Usability Heuristics were used to guide the analysis of data obtained in the intervention development process.

Results

Semistructured qualitative interviews were conducted with 15 primary care Patient Aligned Care Team professionals (13 physicians and 2 clinical pharmacists) at a single VA medical center. Findings highlight that providers were generally comfortable with the paradigm shift to risk-based guidelines but less clear on the need for the VA/DoD guidelines in specific. Providers preferred a clinical decision support tool that helped them calculate patient risk and guide their care without limiting autonomy. They were less comfortable with risk communication and performance measurement systems that do not account for shared decision making. When possible, we incorporated their recommendations into the intervention.

Conclusions

By combining qualitative methods and user-centered design principles, we could inform the design of a multicomponent guideline implementation intervention to better address the needs and preferences of providers, including clear and direct language, logical decision prompts with an option to dismiss a clinical decision support tool, and logical ordering of feedback information. Additionally, this process allowed us to identify future design considerations for quality improvement interventions.

Taking the heat or taking the temperature? A qualitative study of a large-scale exercise in seeking to measure for improvement, not blame

Measurement of quality and safety has an important role in improving healthcare, but is susceptible to unintended consequences. One frequently made argument is that optimising the benefits from measurement requires controlling the risks of blame, but whether it is possible to do this remains unclear. We examined responses to a programme known as the NHS Safety Thermometer (NHS-ST). Measuring four common patient harms in diverse care settings with the goal of supporting local improvement, the programme explicitly eschews a role for blame. The study design was ethnographic. We conducted 115 hours of observation across 19 care organisations and conducted 126 interviews with frontline staff, senior national leaders, experts in the four harms, and the NHS-ST programme leadership and development team. We also collected and analysed relevant documents. The programme theory of the NHS-ST was based in a logic of measurement for improvement: the designers of the programme sought to avoid the appropriation of the data for any purpose other than supporting improvement. However, organisational participants - both at frontline and senior levels - were concerned that the NHS-ST functioned latently as a blame allocation device. These perceptions were influenced, first, by field-level logics of accountability and managerialism and, second, by specific features of the programme, including public reporting, financial incentives, and ambiguities about definitions that amplified the concerns. In consequence, organisational participants, while they identified some merits of the programme, tended to identify and categorise it as another example of performance management, rich in potential for blame. These findings indicate that the search to optimise the benefits of measurement by controlling the risks of blame remains challenging. They further suggest that a well-intentioned programme theory, while necessary, may not be sufficient for achieving goals for improvement in healthcare systems dominated by institutional logics that run counter to the programme theory.

Hospital Perceptions of Medicare's Sepsis Quality Reporting Initiative

BACKGROUND

In October 2015, the Centers for Medicare and Medicaid Services (CMS) implemented the Sepsis CMS Core Measure (SEP-1) program, requiring hospitals to report data on the quality of care for their patients with sepsis.

OBJECTIVE

We sought to understand hospital perceptions of and responses to the SEP-1 program.

DESIGN

A thematic content analysis of semistructured interviews with hospital quality officials.

SETTING

A stratified random sample of short-stay, nonfederal, general acute care hospitals in the United States.

PATIENTS

Hospital quality officers, including nurses and physicians.

MEASUREMENTS

We completed 29 interviews before reaching content saturation.

RESULTS

Hospitals reported a variety of actions in response to SEP-1, including new efforts to collect data, improve sepsis diagnosis and treatment, and manage clinicians' attitudes toward SEP-1. These efforts frequently required dedicated resources to meet the program's requirements for treatment and documentation, which were thought to be complex and not consistently linked to patient-centered outcomes. Most respondents felt that SEP-1 was likely to improve sepsis outcomes. At the same time, they described specific changes that could improve its effectiveness, including allowing hospitals to focus on the treatment processes most directly associated with improved patient outcomes and better aligning the measure's sepsis definitions with current clinical definitions.

CONCLUSIONS

Hospitals are responding to the SEP-1 program across a number of domains and in ways that consistently require dedicated resources. Hospitals are interested in further revisions to the program to alleviate the burden of the reporting requirements and help them optimize the effectiveness of their investments in quality-improvement efforts.

The Implementation of Performance Management Systems in U.S. Physician Organizations

Performance management systems (PMSYS) aid in improving the quality and efficiency of care, but little is known about factors that influence more robust PMSYS among physician organizations. Using a nationally representative survey of U.S. medical practices, we examined the extent to which organizational capabilities and external factors were associated with more developed PMSYS. Linear regression estimated the relative impact of these factors on PMSYS. On average, practices implemented a minority (32 points out of 100) of the PMSYS processes assessed. Practices evaluated ( p < .01) or financially incentivized by external entities ( p < .01), receiving data from health plans ( p < .01), participating in an accountable care organization ( p < .01), affiliating with an independent practice association and/or physician-hospital organization ( p < .01), and using health information technology ( p < .01) and chronic disease registries ( p < .01) to greater degrees had more robust PMSYS. PMSYS of medical practices are underdeveloped, although both external incentives and organizational capabilities may support PMSYS development.