Prioritizing quality indicator development across the healthcare system: identifying what to measure

Defining quality indicators, pharmaceutical care bundles and outcomes of clinical pharmacy service delivery using a Delphi consensus approach

BACKGROUND

Clinical pharmacy quality indicators are often non-uniform and measure individual activities not linked to outcomes.

AIM

To define a consensus agreed pharmaceutical care bundle and patient outcome measures across an entire state health service.

METHOD

A four-round modified-Delphi approach with state Directors of Pharmacy was performed (n = 25). They were asked to rate on a 5-point Likert scale the relevance and measurability of 32 inpatient clinical pharmacy quality indicators and outcome measures. They also ranked clinical pharmacy activities in order from perceived most to least beneficial. Based upon these results, pharmaceutical care bundles consisting of multiple clinical pharmacy activities were formed, and relevance and measurability assessed.

RESULTS

Response rate ranged from 40 to 60%. Twenty-six individual clinical pharmacy quality indicators reached consensus. The top ranked clinical pharmacy quality indicator was 'proportion of patients where a pharmacist documents an accurate list of medicines during admission'. There were nine pharmaceutical care bundles formed consisting between 3 and 7 activities. Only one pharmaceutical care bundle reached consensus: medication history, adverse drug reaction/allergy documentation, admission and discharge medication reconciliation, medication review, provision of medicines education and provision of a medication list on discharge. Sixteen outcome measures reached consensus. The top ranked were hospital acquired complications, readmission due to medication misadventure and unplanned readmission within 10 days.

CONCLUSION

Consensus has been reached on one pharmaceutical care bundle and sixteen outcomes to monitor clinical pharmacy service delivery. The next step is to measure the extent of pharmaceutical care bundle delivery and the link to patient outcomes.

Perioperative quality indicators specific to the practice of anesthesia in noncardiac surgery: an umbrella review

PURPOSE

Improvement in delivery of perioperative care depends on the ability to measure outcomes that can direct meaningful changes in practice. We sought to identify and provide an overview of perioperative quality indicators specific to the practice of anesthesia in noncardiac surgery.

SOURCE

We conducted an umbrella review (a systematic review of systematic reviews) according to Joanna Briggs Institute methodology. We included systematic reviews examining perioperative indicators in patients ≥ 18 yr of age undergoing noncardiac surgery. Our primary outcome was any quality indicator specific to anesthesia. Indicators were classified by the Donabedian system and perioperative phase of care. The quality of systematic reviews was assessed using AMSTAR 2 criteria. Level of evidence of quality indicators was stratified by the Oxford Centre for Evidence-Based Medicine Classification.

PRINCIPAL FINDINGS

Our search returned 1,475 studies. After removing duplicates and screening of abstracts and full texts, 23 systematic reviews encompassing 3,164 primary studies met our inclusion criteria. There were 330 unique quality indicators. Process indicators were most common (n = 169), followed by outcome (n = 114) and structure indicators (n = 47). Few identified indicators were supported by high-level evidence (45/330, 14%). Level 1 evidence supported indicators of antibiotic prophylaxis (1a), venous thromboembolism prophylaxis (1a), postoperative nausea/vomiting prophylaxis (1b), maintenance of normothermia (1a), and goal-directed fluid therapy (1b).

CONCLUSION

This umbrella review highlights the scarcity of perioperative quality indicators that are supported by high quality evidence. Future development of quality indicators and recommendations for outcome measurement should focus on metrics that are supported by level 1 evidence. Potential targets for evidence-based quality-improvement programs in anesthesia are identified herein.

STUDY REGISTRATION

PROSPERO (CRD42020164691); first registered 28 April 2020.

System's Crisis Resilience as a Societal Crisis: Knowledge Structure and Gaze of the Finnish Health Care System

The crisis resilience of vital social systems is currently the target of constant development efforts in Finland, as their drifting into crisis would weaken societies' functional abilities, safety, and security. This is also the case regarding the Finnish health care system. In an attempt to move beyond existing frameworks of crisis imagination, this article takes an unconventional stance by elucidating endogenous crisis dynamics present in the Finnish health care system. Delphi process was conducted for top experts in Finnish health care and crisis management. With a dissensus-seeking orientation, our aim was to fertilize disagreements among panelists to reveal key vulnerabilities in the health system. Despite our efforts to evoke dissensus, the panelists ended up generating a consensus that aims to protect the underlying assumptions of the health system's knowledge structure. Through inductive analysis of expert discourses, the data was analyzed through our research question "what constitutes a crisis-proof health system and a crisis-prone health system". What is framed as a strength of the system by our panelists, namely the ability to maintain legitimacy, improve efficiency, and guarantee continuity, can still have questionable implications that are left ungrasped. A system's theory approach illustrates how such effects can develop and escalate beyond the reach of social interventions, and thus be predisposed to cause objectionable yet concealed social crises. The discussion illustrates how these endogenous crisis dynamics could be seen to materialize in real-life cases.

Primary care quality for older adults: Practice-based quality measures derived from a RAND/UCLA appropriateness method study

We established consensus on practice-based metrics that characterize quality of care for older primary care patients and can be examined using secondary health administrative data. We conducted a two-round RAND/UCLA Appropriateness Method (RAM) study and recruited 10 Canadian clinicians and researchers with expertise relevant to the primary care of elderly patients. Informed by a literature review, the first RAM round evaluated the appropriateness and importance of candidate quality measures in an online questionnaire. Technical definitions were developed for each endorsed indicator to specify how the indicator could be operationalized using health administrative data. In a virtual synchronous meeting, the expert panel offered feedback on the technical specifications for the endorsed indicators. Panelists then completed a second (final) questionnaire to rate each indicator and corresponding technical definition on the same criteria (appropriateness and importance). We used statistical integration to combine technical expert panelists' judgements and content analysis of open-ended survey responses. Our literature search and internal screening resulted in 61 practice-based quality indicators for rating. We developed technical definitions for indicators endorsed in the first questionnaire (n = 55). Following the virtual synchronous meeting and second questionnaire, we achieved consensus on 12 practice-based quality measures across four Priority Topics in Care of the Elderly. The endorsed indicators provide a framework to characterize practice- and population-level encounters of family physicians delivering care to older patients and will offer insights into the outcomes of their care provision. This study presented a case of soliciting expert feedback to develop measurable practice-based quality indicators that can be examined using administrative data to understand quality of care within population-based data holdings. Future work will refine and operationalize the technical definitions established through this process to examine primary care provision for older adults in a particular context (Ontario, Canada).

Development of practice-based quality indicators for the primary care of older adults: a RAND/UCLA Appropriateness Method study protocol

INTRODUCTION

Older adults have high rates of primary care utilisation, and quality primary care has the potential to address their complex medical needs. Family physicians have different levels of knowledge and skills in caring for older patients, which may influence the quality of care delivery and resulting health outcomes. In this study, we aim to establish consensus on practice-based metrics that characterise quality of care for older primary care patients and can be examined using secondary, administrative data.

METHODS AND ANALYSIS

We describe a two-round RAND/UCLA Appropriateness Method (RAM) study to assess the consensus of a technical expert panel. We will recruit pan-Canadian experts who demonstrate excellence in clinical practice or scholarship related to the primary care of older adults. A literature review will generate a candidate list of practice-based quality indicators. The first round aims to evaluate the appropriateness and importance of candidate indicators through an online questionnaire. We will then develop technical definitions for each endorsed indicator using ICES data holdings. Panellists will offer feedback on the technical definitions in a virtual synchronous meeting and provide ratings on the same criteria in a second questionnaire.

ETHICS AND DISSEMINATION

Our study has been approved by the Hamilton Integrated Research Ethics Board (Project ID #15545). Findings will be disseminated via manuscripts, presentations and the lead author's thesis.

TRIAL REGISTRATION NUMBER

ISRCTN17074347.

Development and validation of an audit tool for fluid management in non-critically ill adults in the emergency department

Especially in the emergency department (ED), it is critical to identify weaknesses in prescribing behavior of IV maintenance fluids to ensure a qualitative 24-h fluid management plan. The primary aim of this study was to develop an audit instrument to assess the pitfalls in documentation and prescribing habits of IV fluid therapy for non-critically ill patients admitted to the ED. In this study, an expert panel initially designed the tool. During different phases, adaptations were made to optimize inter-rater agreement (Fleiss' kappa, κ) and validity was determined based on the application of the tool on randomly selected electronic ED patient records. Second, the IV fluid prescription's appropriateness was addressed. The final tool consists of three parts: fluid status assessment, evaluation of maintenance fluid needs and, if present, a limited appropriateness check of the fluid prescription. A manual enhanced inter-rater agreement. κ-values achieved the target value (0.40) after three adaptation rounds, except for the category of 'clinical observations' in the assessment part (κ = 0.531, 95% CI 0.528-0.534), which was acceptable. A check of the IV fluid's indication and volume was only possible with an additional expert evaluation to conclude on prescription appropriateness. Criterion related validity of the final version was high (93.4%). To conclude, the instrument is considered reliable and can be used in clinical practice to evaluate ED fluid management. Thorough documentation is essential to evaluate the appropriateness of the IV fluid prescription, to improve information transfer on IV fluid therapy to the ward and to facilitate retrospective chart review of ED prescribing behavior.

Purpose, Subject, and Consumer Comment on "Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study"

Zegers and colleagues' study codifies the perceived burden of quality monitoring and improvement stemming from the work by clinicians of registering (documenting) quality information in the medical record. We agree with Zegers and colleagues' recommendation that a smaller, more effective and curated set of measures is needed to reduce burden, confusion, and expense. We further note that focusing on validity of clinical evidence behind individual measures is critical, but insufficient. We therefore extend Zegers and colleagues' work through a pragmatic, tripartite heuristic. To assess the value of and curate a targeted set of performance measures, we propose concentrating on the relationships among three factors: (1) The purpose of the performance measure, (2) the subject being evaluated, and (3) the consumer using information for decision-making. Our proposed tripartite framework lays the groundwork for executing the evidence-based recommendations proposed by Zegers et al, and provides a path forward for more effective healthcare performance-measurement systems.

Selection of Pediatric Mental Health Quality Measures for Health System Improvement in British Columbia Based on a Modified Delphi Approach

BACKGROUND

The COVID-19 pandemic has highlighted the importance of mental wellbeing. The identification and implementation of quality measures can improve health outcomes and patient experience. The objective was to identify and define a core set of valid and relevant pediatric mental health quality measures that will support health system evaluation and quality improvement in British Columbia, Canada.

METHODS

The study consisted of four phases. First, a comprehensive database search identified valid pediatric quality measures focused on mental health and substance use (MH/SU). Second, the identified quality measures were mapped to focus areas, which were then prioritized by two stakeholder groups consisting of 26 members. Third, up to two representative measures for each prioritized focus area were pre-selected by an expert panel (n = 9). And fourth, a three-step modified Delphi approach was employed to (1) assess each quality measure on a 7-point Likert scale against three relevance criteria (representative of a quality problem, value to intended audience and actionable), (2) discuss the results, and (3) select and rank the most relevant measures. Forty-eight stakeholders were invited to participate; of those 24 completed the round 1 survey, 21 participated in the round 2 discussion and 18 voted in the round 3 selection and ranking survey. For round 1, consensus was determined when at least 70% of the response rates were within the range of five to seven. For round 3, Kendall's coefficient of concordance W was used as an estimator of inter-rater reliability.

RESULTS

One-hundred pediatric mental health quality measures were identified in the database search. Of those, 37 were mapped to ten focus areas. Pre-selection resulted in 19 representative measures moving forward to the Delphi study. Eleven measures met the consensus thresholds and were brought forward to the round 2 discussion. Round 3 ranking showed moderate to strong raters' agreement (Kendall's W = 0.595; p < 0.01) and resulted in the following five highest-ranked measures: level of satisfaction after discharge from inpatient admission due to MH/SU, number of patients experiencing seclusion or restraint, length of time from eating disorder referral to assessment, number of ED visits due to MH/SU, and number of readmissions to ED.

CONCLUSION

The selected core set of valid and relevant pediatric quality measures will support sustainable system change in British Columbia. The five top-ranked measures will be refined and tested for data collection feasibility before being implemented in the province.

Indicators for local health plan monitoring and evaluation: A modified Delphi consensus

OBJECTIVE

There is still a lack of health indicators for monitoring and evaluating health planning at the local level. In Portugal, local health plans (LHP) include a prioritized set of health priorities, which should be monitored and evaluated. This study is an example of a low-resource method to identify and reuse indicators for LHP monitoring and evaluation already collected for other purposes.

DESIGN AND SAMPLE

A modified Delphi consensus method was applied, with three rounds of email rating questionnaires and a final meeting, between January 2018 and January 2019. The Delphi panel consisted of eight members from the Planning and Administration Group of the Espinho/Gaia Local Public Health Unit.

MEASUREMENTS

Panelists were asked to assess the indicators' validity for monitoring diseases/determinants from a pre-selected list of potential binomials between 140 PHC indicators and 15 diseases/determinants.

RESULTS

After four rounds, there was consensus in considering 141 binomials (34.0%) as appropriate, diabetes mellitus being the disease with more appropriate indicators.

CONCLUSION

This study portrays the applicability of a commonly used, easy and low-resource method in a Portuguese Local Public Health Unit to select and reuse primary health care indicators for LHP monitoring and evaluation.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

Assessing the Quality of the Systems of Care for Children with Congenital Zika Virus Infection and Other Neurodevelopmental Disabilities in the United States Pacific Island Territories

Congenital Zika virus (ZIKV) infection can cause lifelong medical and developmental conditions and management needs. There is limited information on the strengths and weaknesses of the systems of care for addressing ZIKV and other neurodevelopmental disabilities (NRD) in the United States (US) Affiliated Pacific Island Territories. Therefore, the purpose of the study was to assess the quality of the chronic illness systems of care for children with congenital ZIKV and other NRD in the US Pacific Island Territories. A cross-sectional study was conducted among health professionals from American Samoa, Guam, and Commonwealth of the Northern Mariana Islands. Participants completed an adapted version of the Assessment of Chronic Illness Care 3.5 (ACIC), which is based on the Chronic Care Model. The median Total Program Score was calculated, which ranged from limited support (0-2), basic support (3-5), reasonably good support (6-8), to fully developed support for care (9-11). Among the 17 health professionals who completed the survey, 47% were Guamanian/Chamorro, 24% were Samoan, 12% were Filipino, and 6% were Other Pacific Islanders. The median (25th percentile, 75th percentile [interquartile range]) Total Program Score was 5 (3, 6 [3]), indicating basic support for ZIKV and other NRD care for children. As more is learned about the full spectrum of clinical findings related to ZIKV, it is critical to continue to build an interdisciplinary maternal and child health workforce with the capacity and preparation to adequately address the special needs of children with ZIKV and other NRD.

Clinical indicators for reporting the effectiveness of patient quality and safety-related interventions: a protocol of a systematic review and Delphi consensus process as part of the international Standardised Endpoints for Perioperative Medicine initiative (StEP)

INTRODUCTION

Clinical indicators are used to measure and quantify the safety and quality of patient care. They are also often used as endpoints in clinical trials. Definitions of clinical indicators in common use are extremely heterogeneous, limiting their applicability. As part of the international Standardised Endpoints in Perioperative Medicine initiative, this study will identify clinical indicators by systematically reviewing the anaesthesia and perioperative medicine literature, and will provide consensus, clinically useful definitions for those indicators using a Delphi process.

METHODS AND ANALYSIS

An electronic database search will be conducted of Medline (PubMed/OVID), EMBASE and the Cochrane Library in order to meet this review's objectives that are: (1) To identify clinical indicators and their definitions used in randomised controlled trials that assess patient-related quality and safety interventions in perioperative medicine; (2) To select a shortlist of recommended indicators and definitions that are the most suitable for evaluation of quality and safety interventions following an expert-based consensus-gaining process (Delphi method) and (3) To provide a classification scale for each indicator related to its clarity of definition, validity (strength), reliability, feasibility (ease of use) and frequency of use. This systematic review protocol is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols guidance.

ETHICS AND DISSEMINATION

Ethical approval is not required for this systematic review and Delphi process. The results of this study will be disseminated to the anaesthesia and perioperative medicine clinical and academic community through national and international presentations and through publication in a peer reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42016042102.

Quality indicators for responsible use of medicines: a systematic review

OBJECTIVE

All healthcare systems require valid ways to evaluate service delivery. The objective of this study was to identify existing content validated quality indicators (QIs) for responsible use of medicines (RUM) and classify them using multiple frameworks to identify gaps in current quality measurements.

DESIGN

Systematic review without meta-analysis.

SETTING

All care settings.

SEARCH STRATEGY

CINAHL, Embase, Global Health, International Pharmaceutical Abstract, MEDLINE, PubMed and Web of Science databases were searched up to April 2018. An internet search was also conducted. Articles were included if they described medication-related QIs developed using consensus methods. Government agency websites listing QIs for RUM were also included.

ANALYSIS

Several multidimensional frameworks were selected to assess the scope of QI coverage. These included Donabedian's framework (structure, process and outcome), the Anatomical Therapeutic Chemical (ATC) classification system and a validated classification for causes of drug-related problems (c-DRPs; drug selection, drug form, dose selection, treatment duration, drug use process, logistics, monitoring, adverse drug reactions and others).

RESULTS

2431 content validated QIs were identified from 131 articles and 5 websites. Using Donabedian's framework, the majority of QIs were process indicators. Based on the ATC code, the largest number of QIs pertained to medicines for nervous system (ATC code: N), followed by anti-infectives for systemic use (J) and cardiovascular system (C). The most common c-DRPs pertained to 'drug selection', followed by 'monitoring' and 'drug use process'.

CONCLUSIONS

This study was the first systematic review classifying QIs for RUM using multiple frameworks. The list of the identified QIs can be used as a database for evaluating the achievement of RUM. Although many QIs were identified, this approach allowed for the identification of gaps in quality measurement of RUM. In order to more effectively evaluate the extent to which RUM has been achieved, further development of QIs may be required.

[Identifying indicators of good practice in clinical and healthcare management]

OBJECTIVE

To identify good practices in order to develop and implement indicators of health outcomes for clinical and healthcare management, as well as the characteristics for an indicator to be considered adequate.

METHODOLOGY

A scoping review was performed, with the following phases: 1) Search and identification of bibliography. 2) Selection of relevant documents. Including those studies that discussed issues related to good practices for the use of health indicators in the management field. Those published in a language other than English or Spanish or before 2006 were excluded. 3) Analysis and extraction of information. 4) Consultation with stakeholders, using a qualitative methodology through Concept Mapping, with the participation of 40 experts (decision-makers, scientific societies, and health professionals). The data collection process included an inductive and structured procedure, with prioritisation of ideas grouped into clusters, according to feasibility and importance criteria (0-10 scale).

RESULTS

Good practices identified 2 levels: 1) macro-management: Define a framework for the evaluation of indicators and establish a benchmark of indicators. 2) meso-management: Establish indicators according to evidence and expert consensus, taking into account priority areas and topics, testing before final use, and communicate results adequately. The characteristics of a suitable indicator are: 1) Approach of an important issue, 2) Scientific validity, 3) Possibility of measurement with reliable data, 4) Meaning of useful and applicable measurement, and 5) Wide scope.

CONCLUSIONS

The best practices for the use of indicators in clinical and healthcare management can make it easier to monitor performance and accountability, as well as to support the decision-making addressed at the development of initiatives for quality improvement.

Perceptions and attitudes towards the implementation of a disability evaluation system based on the international classification of functioning, disability, and health among people with disabilities in Taiwan

PURPOSE

To explore the perceptions and attitudes among people with disabilities towards the newly implemented International Classification of Functioning, Disability and Health-based disability evaluation system (the new system) in Taiwan.

METHOD

Using a self-administered questionnaire, we conducted a nationwide survey. The questionnaire focused on the domains of quality, satisfaction, and revision of the new system. In total, 1073 persons (age, ≥18 years) with disabilities or their primary caregivers, who experienced both the old and the new system, responded to the questionnaire.

RESULTS

Most participants were satisfied with the new system overall (58.7%) and the subscale of quality of structure (91.3%) and quality of outcome (63.6%). However, only 20.5% of the participants were favourable to the quality of process. The probability of being satisfied with the system overall was low for the quality of process subscale (adjusted odds ratio range, 0.3 ∼ 0.4) and its item of long interval (0.2 ∼ 0.6). Contrariwise, the probability was high for the other subscales (3.9 ∼ 13.7) and the item of identifying needs (21.9 ∼ 23.4).

CONCLUSIONS

Persons with disabilities and their primary caregivers have positive attitudes towards the new system. It is important to simplify the assessment tools and procedures to improve the system's quality of process and facilitate its usability. IMPLICATIONS FOR REHABILITATION Persons with disabilities have positive attitudes towards the newly implemented International Classification of Functioning, Disability, and Health-based disability evaluation system in Taiwan. The system that provides comprehensive information about functioning and disability of persons with disabilities is able to capture the difficulties and needs of those individuals in their daily lives. The system hence helps people to mitigate the effects of disability and guide rehabilitation. The assessment items and processes of the system, however, were perceived to be complicated, time-consuming, and inconvenient. Simplifying the assessment items and processes, such as developing a short form version of the assessment tool and increasing the service time, may facilitate the usability of the system.

Are performance indicators used for hospital quality management: a qualitative interview study amongst health professionals and quality managers in The Netherlands

BACKGROUND

Hospitals are under increasing pressure to share indicator-based performance information. These indicators can also serve as a means to promote quality improvement and boost hospital performance. Our aim was to explore hospitals' use of performance indicators for internal quality management activities.

METHODS

We conducted a qualitative interview study among 72 health professionals and quality managers in 14 acute care hospitals in The Netherlands. Concentrating on orthopaedic and oncology departments, our goal was to gain insight into data collection and use of performance indicators for two conditions: knee and hip replacement surgery and breast cancer surgery. The semi-structured interviews were recorded and summarised. Based on the data, themes were synthesised and the analyses were executed systematically by two analysts independently. The findings were validated through comparison.

RESULTS

The hospitals we investigated collect data for performance indicators in different ways. Similarly, these hospitals have different ways of using such data to support their quality management, while some do not seem to use the data for this purpose at all. Factors like 'linking pin champions', pro-active quality managers and engaged medical specialists seem to make a difference. In addition, a comprehensive hospital data infrastructure with electronic patient records and robust data collection software appears to be a prerequisite to produce reliable external performance indicators for internal quality improvement.

CONCLUSIONS

Hospitals often fail to use performance indicators as a means to support internal quality management. Such data, then, are not used to its full potential. Hospitals are recommended to focus their human resource policy on 'linking pin champions', the engagement of professionals and a pro-active quality manager, and to invest in a comprehensive data infrastructure. Furthermore, the differences in data collection processes between Dutch hospitals make it difficult to draw comparisons between outcomes of performance indicators.

Quality in paediatric emergency medicine: Measurement and reporting

There is a clear demand for quality in the delivery of health care around the world; paediatric emergency medicine is no exception to this movement. It has been identified that gaps exist in the quality of acute care provided to children. Regulatory bodies in Australia and New Zealand are moving to mandate the implementation of quality targets and measures. Within the paediatric emergency department (ED), there is a lack of research into paediatric specific indicators. The existing literature regarding paediatric acute care quality measures has been recently summarised, and expert consensus has now been reported. It is clear that there is much work to be performed to generalise this work to ED. We review suggestions from the current literature relating to feasible indicators within the paediatric acute care setting. We propose options to develop a quality 'scorecard' that could be used to assist Australian and New Zealand EDs with quality measurement and benchmarking for their paediatric patients.

Quality of HIV Care and Mortality Rates in HIV-Infected Patients

BACKGROUND

The Patient Protection and Affordable Care Act encourages healthcare systems to track quality-of-care measures; little is known about their impact on mortality rates. The objective of this study was to assess associations between HIV quality of care and mortality rates.

METHODS

A longitudinal survival analysis of the Veterans Aging Cohort Study included 3038 human immunodeficiency virus (HIV)-infected patients enrolled between June 2002 and July 2008. The independent variable was receipt of ≥80% of 9 HIV quality indicators (QIs) abstracted from medical records in the 12 months after enrollment. Overall mortality rates through 2014 were assessed from the Veterans Health Administration, Medicare, and Social Security National Death Index records. We assessed associations between receiving ≥80% of HIV QIs and mortality rates using Kaplan-Meier survival analysis and adjusted Cox proportional hazards models. Results were stratified by unhealthy alcohol and illicit drug use.

RESULTS

The majority of participants were male (97.5%) and black (66.8%), with a mean (standard deviation) age of 49.0 (8.8) years. Overall, 25.9% reported past-year unhealthy alcohol use and 28.4% reported past-year illicit drug use. During 24 805 person-years of follow-up (mean [standard deviation], 8.2 [3.3] years), those who received ≥80% of QIs experienced lower age-adjusted mortality rates (adjusted hazard ratio, 0.75; 95% confidence interval, .65-.86). Adjustment for disease severity attenuated the association.

CONCLUSIONS

Receipt of ≥80% of select HIV QIs is associated with improved survival in a sample of predominantly male, black, HIV-infected patients but was insufficient to overcome adjustment for disease severity. Interventions to ensure high-quality care and address underlying chronic illness may improve survival in HIV-infected patients.

Evaluation of a continuous monitoring and feedback initiative to improve quality of anaesthetic care: a mixed-methods quasi-experimental study

Background

This study evaluated the impact of a continuous quality monitoring and feedback initiative in anaesthesia.

Objectives

To conduct a quasi-experimental evaluation of the feedback initiative and its effect on quality of anaesthetic care and perioperative efficiency. To understand the longitudinal effects of passive and active feedback and investigate the mechanisms and interactions underpinning those effects.

Design

Mixed-methods evaluation with analysis and synthesis of data from longitudinal qualitative interviews, longitudinal evaluative surveys and an interrupted time series study.

Intervention

Continuous measurement of a range of anaesthetic quality indicators was undertaken in a London teaching hospital alongside monthly personal feedback from case summary data to a cohort of anaesthetists, with follow-up roll-out to the whole NHS trust. Basic feedback consisted of the provision of passive monthly personalised feedback reports containing summary case data. In the enhanced phase, data feedback consisted of more sophisticated statistical breakdown of data, comparative and longitudinal views, and was paired with an active programme of dissemination and professional engagement.

Methods

Baseline data collection began in March 2010. Implementation of basic feedback took place in October 2010, followed by implementation of the enhanced feedback protocol in July 2012. Weekly aggregated quality indicator data, coupled with surgical site infection and mortality rates, was modelled using interrupted time series analyses. The study anaesthetist cohort comprised 50,235 cases, performed by 44 anaesthetists over the course of the study, with 22,670 cases performed at the primary site. Anaesthetist responses to the surveys were collected pre and post implementation of feedback at all three sites in parallel with qualitative investigation. Seventy anaesthetists completed the survey at one or more time points and 35 health-care professionals, including 24 anaesthetists, were interviewed across two time points.

Results

Results from the time series analysis of longitudinal variation in perioperative indicators did not support the hypothesis that implementation of basic feedback improved quality of anaesthetic care. The implementation of enhanced feedback was found to have a significant positive impact on two postoperative pain measures, nurse-recorded freedom from nausea, mean patient temperature on arrival in recovery and Quality of Recovery Scale scores. Analysis of survey data demonstrated that anaesthetists value perceived credibility of data and local relevance of quality indicators above other criteria when assessing utility of feedback. A significant improvement in the perceived value of quality indicators, feedback, data use and overall effectiveness was observed between baseline and implementation of feedback at the primary site, a finding replicated at the two secondary sites. Findings from the qualitative research elucidated processes of interaction between context, intervention and user, demonstrating a positive response by clinicians to this type of initiative and willingness to interact with a sustained and comprehensive feedback protocol to understand variations in care.

Conclusions

The results support the potential of quality monitoring and feedback interventions as quality improvement mechanisms and provide insight into the positive response of clinicians to this type of initiative, including documentation of the experiences of anaesthetists that participated as users and codesigners of the feedback. Future work in this area might usefully investigate how this type of intervention may be transferred to other areas of clinical practice and further explore interactions between local context and the successful implementation of quality monitoring and feedback systems.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

[Warning systems in a computerized nursing process for Intensive Care Units]

A hybrid study combining technological production and methodological research aiming to establish associations between the data and information that are part of a Computerized Nursing Process according to the ICNP® Version 1.0, indicators of patient safety and quality of care. Based on the guidelines of the Agency for Healthcare Research and Quality and the American Association of Critical Care Nurses for the expansion of warning systems, five warning systems were developed: potential for iatrogenic pneumothorax, potential for care-related infections, potential for suture dehiscence in patients after abdominal or pelvic surgery, potential for loss of vascular access, and potential for endotracheal extubation. The warning systems are a continuous computerized resource of essential situations that promote patient safety and enable the construction of a way to stimulate clinical reasoning and support clinical decision making of nurses in intensive care.

Quality of pharmaceutical care in surgical patients

Background

Surgical patients are at risk for preventable adverse drug events (ADEs) during hospitalization. Usually, preventable ADEs are measured as an outcome parameter of quality of pharmaceutical care. However, process measures such as QIs are more efficient to assess the quality of care and provide more information about potential quality improvements.

Objective

To assess the quality of pharmaceutical care of medication-related processes in surgical wards with quality indicators, in order to detect targets for quality improvements.

Methods

For this observational cohort study, quality indicators were composed, validated, tested, and applied on a surgical cohort. Three surgical wards of an academic hospital in the Netherlands (Academic Medical Centre, Amsterdam) participated. Consecutive elective surgical patients with a hospital stay longer than 48 hours were included from April until June 2009. To assess the quality of pharmaceutical care, the set of quality indicators was applied to 252 medical records of surgical patients.

Results

Thirty-four quality indicators were composed and tested on acceptability and content- and face-validity. The selected 28 candidate quality indicators were tested for feasibility and ‘sensitivity to change’. This resulted in a final set of 27 quality indicators, of which inter-rater agreements were calculated (kappa 0.92 for eligibility, 0.74 for pass-rate). The quality of pharmaceutical care was assessed in 252 surgical patients. Nearly half of the surgical patients passed the quality indicators for pharmaceutical care (overall pass rate 49.8%). Improvements should be predominantly targeted to medication care related processes in surgical patients with gastro-intestinal problems (domain pass rate 29.4%).

Conclusions

This quality indicator set can be used to measure quality of pharmaceutical care and detect targets for quality improvements. With these results medication safety in surgical patients can be enhanced.

Development of a quality ranking model for home health care providers

This research aims to increase transparency and simplify consumer decision-making regarding the selection of a home health care provider. Currently, quality information on home health care providers is fragmented and difficult to interpret. In this study, a quality-ranking model is developed by selecting multidimensional quality indicators across multiple sources and respective weights using expert judgment. Given the weights and providers' performance on each quality indicator, a composite score is calculated that summarizes a home health care provider's overall quality level. This quality information empowers consumers to narrow their search and select the best-performing, most efficient providers.

Contribuições dos registros eletrônicos para a segurança do paciente em terapia intensiva: uma revisão integrativa

Revisão integrativa que analisou nas publicações as contribuições dos registros eletrônicos em saúde para a segurança do paciente em unidades de terapia intensiva. A pesquisa foi realizada nas bases de dados CINAHL, MEDLINE e SciELO, utilizando os descritores: registros eletrônicos de saúde, sistemas de informação, informática em enfermagem, informática médica, unidades de terapia intensiva, segurança do paciente e gerenciamento de segurança. Foram incluídos 64 artigos, sendo analisados em três categorias: "sistemas de informação e informática em saúde: o registro eletrônico para a continuidade do cuidado de Enfermagem", "sistemas de apoio à decisão: contribuições para a segurança do paciente" e "indicadores de qualidade do cuidado e de segurança do paciente partir dos registros eletrônicos". Os estudos apontaram como contribuições a continuidade do cuidado, a tomada de decisão baseada nos sistemas de apoio à decisão e a criação de indicadores de qualidade e segurança do paciente a partir dos registros eletrônicos.

The Prostate Cancer Registry: monitoring patterns and quality of care for men diagnosed with prostate cancer

OBJECTIVE

To establish a pilot population-based clinical registry with the aim of monitoring the quality of care provided to men diagnosed with prostate cancer.

PATIENTS AND METHODS

All men aged >18 years from the contributing hospitals in Victoria, Australia, who have a diagnosis of prostate cancer confirmed by histopathology report notified to the Victorian Cancer Registry are eligible for inclusion in the Prostate Cancer Registry (PCR). A literature review was undertaken aiming to identify existing quality indicators and source evidence-based guidelines from both Australia and internationally.

RESULTS

A Steering Committee was established to determine the minimum dataset, select quality indicators to be reported back to clinicians, identify the most effective recruitment strategy, and provide a governance structure for data requests; collection, analysis and reporting of data; and managing outliers. A minimum dataset comprising 72 data items is collected by the PCR, enabling ten quality indicators to be collected and reported. Outcome measures are risk adjusted according to the established National Comprehensive Cancer Network and Cancer of the Prostate Risk Assessment Score (surgery only) risk stratification model. Recruitment to the PCR occurs concurrently with mandatory notification to the state-based Cancer Registry. The PCR adopts an opt-out consent process to maximize recruitment. The data collection approach is standardized, using a hybrid of data linkage and manual collection, and data collection forms are electronically scanned into the PCR. A data access policy and escalation policy for mortality outliers has been developed.

CONCLUSIONS

The PCR provides potential for high-quality population-based data to be collected and managed within a clinician-led governance framework. This approach satisfies the requirement for health services to establish quality assessment, at the same time as providing clinically credible data to clinicians to drive practice improvement.

Validation of prescribing appropriateness criteria for older Australians using the RAND/UCLA appropriateness method

OBJECTIVE

To further develop and validate previously published national prescribing appropriateness criteria to assist in identifying drug-related problems (DRPs) for commonly occurring medications and medical conditions in older (≥65 years old) Australians.

DESIGN

RAND/UCLA appropriateness method.

PARTICIPANTS

A panel of medication management experts were identified consisting of geriatricians/pharmacologists, clinical pharmacists and disease management advisors to organisations that produce Australian evidence-based therapeutic publications. This resulted in a round-one panel of 15 members, and a round-two panel of 12 members.

MAIN OUTCOME MEASURE

Agreement on all criteria.

RESULTS

Forty-eight prescribing criteria were rated. In the first rating round via email, there was disagreement regarding 17 of the criteria according to median panel ratings. During a face-to-face second round meeting, discussion resulted in retention of 25 criteria after amendments, agreement for 14 criteria with no changes required and deletion of 9 criteria. Two new criteria were added, resulting in a final validated list of 41 prescribing appropriateness criteria. Agreement after round two was reached for all 41 criteria, measured by median panel ratings and the amount of dispersion of panel ratings, based on the interpercentile range.

CONCLUSIONS

A set of 41 Australian prescribing appropriateness criteria were validated by an expert panel. Use of these criteria, together with clinical judgement and other medication review processes such as patient interview, is intended to assist in improving patient care by efficiently detecting potential DRPs related to commonly occurring medicines and medical conditions in older Australians. These criteria may also contribute to the medication management education of healthcare professionals.

Quality in the provision of headache care. 2: defining quality and its indicators

The objective of this study was to define “quality” of headache care, and develop indicators that are applicable in different settings and cultures and to all types of headache. No definition of quality of headache care has been formulated. Two sets of quality indicators, proposed in the US and UK, are limited to their localities and/or specific to migraine and their development received no input from people with headache. We first undertook a literature review. Then we conducted a series of focus-group consultations with key stakeholders (doctors, nurses and patients) in headache care. From the findings we proposed a large number of putative quality indicators, and refined these and reduced their number in consultations with larger international groups of stakeholder representatives. We formulated a definition of quality from the quality indicators. Five main themes were identified: (1) headache services; (2) health professionals; (3) patients; (4) financial resources; (5) political agenda and legislation. An initial list of 160 putative quality indicators in 14 domains was reduced to 30 indicators in 9 domains. These gave rise to the following multidimensional definition of quality of headache care: “Good-quality headache care achieves accurate diagnosis and individualized management, has appropriate referral pathways, educates patients about their headaches and their management, is convenient and comfortable, satisfies patients, is efficient and equitable, assesses outcomes and is safe.” Quality in headache care is multidimensional and resides in nine essential domains that are of equal importance. The indicators are currently being tested for feasibility of use in clinical settings.

Using quality indicators in anaesthesia: feeding back data to improve care

After recent UK policy developments, considerable attention has been focused upon how clinical specialties measure and report on the quality of care delivered to patients. Defining the right indicators alone is insufficient to close the feedback loop. This narrative review aims to describe and synthesize a diverse body of research relevant to the question of how information from quality indicators can be fed back and used effectively to improve care. Anaesthesia poses certain challenges in the identification of valid outcome indicators sensitive to variations in anaesthetic care. Metrics collected during the immediate post-anaesthetic recovery period, such as patient temperature, patient-reported quality of recovery, and pain and nausea, provide potentially useful information for the anaesthetist, yet this information is not routinely fed back. Reviews of the effects of feeding back performance data to healthcare providers suggest that this may result in small to moderate positive effects upon outcomes and professional practice, with stronger effects where feedback is integrated within a broader quality improvement strategy. The dominant model for use of data within quality improvement is based upon the industrial process control approach, in which care processes are monitored continuously for process changes which are rapidly detectable for corrective action. From this review and experience of implementing these principles in practice, effective feedback from quality indicators is timely, credible, confidential, tailored to the recipient, and continuous. Considerable further work is needed to understand how information from quality indicators can be fed back in an effective way to clinicians and clinical units, in order to support revalidation and continuous improvement.

An international environmental scan of quality indicators for cardiovascular care

Quality indicators (QIs) are increasingly being used to measure and improve the quality of cardiac care. We conducted an international environmental scan to identify and critically appraise published QI development initiatives addressing cardiovascular disease (CVD). A review of the peer-reviewed and grey English-language literature was conducted to identify published CVD QI development initiatives. The quality of identified studies was assessed using a modified version of the Appraisal of Guidelines for Research and Evaluation (AGREE) II QI tool-an instrument originally developed for the assessment of the quality of clinical practice guidelines. An initial literature search identified 2314 potentially relevant abstracts of peer-reviewed articles. After a review of the abstracts, 120 full text articles were retrieved and reviewed. Of these, 20 articles and 1 peer-reviewed monograph were selected for critical appraisal (n = 21). Most of the initiatives were conducted in North America (76%) and were published after 2005 (62%). The majority (5 of 6) of the AGREE II QI domain scores were skewed toward higher values, including the median score for the 'overall quality' rating (83.3%). Of the CVD categories addressed within the 21 initiatives, heart failure was the most common (n = 10 QI indicator sets), followed by acute coronary syndromes (n = 8). Considerable variation was observed in the methods utilized and the degree of scientific rigour applied in the published international CVD QI development initiatives. Adoption of standardized methods could help improve the quality of QI development initiatives.

"Death in low-mortality diagnosis-related groups": frequency, and the impact of patient and hospital characteristics

OBJECTIVE

To examine the frequency of deaths in low-mortality diagnosis-related groups (LM-DRGs) and the patient and hospital characteristics associated with them.

DESIGN, SETTING AND PATIENTS

Retrospective cohort study of 2,400,089 discharge episodes for adults (> 18 years) from 122 Victorian public hospitals from 1 July 2006 to 30 June 2008.

MAIN OUTCOME MEASURES

Frequency of episodes of death in LM-DRGs (defined as DRGs with mortality < 0.5% over the previous 3 years or < 0.5% in any of the previous 3 years); associations between characteristics of patients and hospitals with deaths in LM-DRGs.

RESULTS

There were 1,008, 816 LM-DRG episodes with 0-15 LM-DRG deaths per hospital in the 2006-07 financial year and 0-20 deaths per hospital in the 2007-08 financial year. Increased age, level of comorbidity, being male, admission from a residential aged care facility, interhospital transfer, emergency admission and lower hospital volume were associated with an increased risk of death in LM-DRG episodes in both years. Metropolitan location and teaching/major provider status were not associated with LM-DRG deaths (P > 0.10). More than 40% of LM-DRG deaths were among patients aged 83 years or over, who had a length of stay of less than 1 day and had a medical DRG classification. Standardised mortality ratios (SMRs) that adjusted for the patient and hospital characteristics identified nine outlier hospitals with high frequencies of deaths in LM-DRGs in the 2006-07 and six in the 2007-08 financial year compared with 59 hospitals flagged by the death-in-LM-DRG indicator.

CONCLUSIONS

The use of the LM-DRG indicator requires further investigation to test its validity. LM-DRG deaths are infrequent, making it difficult to identify temporal changes and outlier hospitals. Patient characteristics unrelated to quality of care increase the likelihood of death among LM-DRG patients. The SMR analysis showed that failure to adjust for these characteristics may result in unfair and inaccurate identification of outlier hospitals. The increased risk of death associated with interhospital transfer patients and low-volume hospitals requires further investigation.