Measuring quality of care: considering measurement frameworks and needs assessment to guide quality indicator development

Caregiver-specific quality measures for home- and community-based services: Environmental scan and stakeholder priorities

Although family caregivers are increasingly recognized for their essential role in helping vulnerable adults live in the community for as long as possible, their priorities and perspectives have not been well-integrated into quality assessments of home- and community-based services (HCBS). Our overall goal was to identify measurement gaps to guide monitoring and improve HCBS. Caregiver-specific measurement priorities were identified during a multi-level stakeholder engagement process that included 34 Veterans, 24 caregivers, and 39 facility leaders, clinicians, and staff across four VA healthcare systems. We mapped items from national quality measure sets for HCBS identified during an environmental scan onto the stakeholder-identified measurement priorities. Only 5 of 11 non-VA measure sets and three of four VA measure sets explicitly included caregiver-specific items that were aligned with or relevant to stakeholders' measurement priorities. Six of 14 stakeholder-identified priorities were not reflected in any measure sets, such as those that explicitly assess caregiver-reported experience with services that directly or indirectly support their role as caregivers within HCBS. Although family caregivers fulfill a critical role in helping adults with complex medical needs live independently for as long as possible, their priorities and perspectives have not been well-integrated into quality assessments of HCBS. Measures that acknowledge caregivers' roles and incorporate their priorities can help healthcare systems to better monitor and improve HCBS quality, thereby enabling Veterans to remain in the community as long as possible.

Follow-up after major traumatic injury: a survey of services in Australian and New Zealand public hospitals

BACKGROUND

Increased survival from traumatic injury has led to a higher demand for follow-up care when patients are discharged from hospital. It is currently unclear how follow-up care following major trauma is provided to patients, and how, when, and to whom follow-up services are delivered. The aim of this study was to describe the current follow-up care provided to patients and their families who have experienced major traumatic injury in Australia and New Zealand (ANZ).

METHODS

Informed by Donabedian's 'Evaluating the Quality of Medical Care' model and the Institute of Medicine's Six Domains of Healthcare Quality, a cross-sectional online survey was developed in conjunction with trauma experts. Their responses informed the final survey which was distributed to key personnel in 71 hospitals in Australia and New Zealand that (i) delivered trauma care to patients, (ii) provided data to the Australasian Trauma Registry, or (iii) were a Trauma Centre.

RESULTS

Data were received from 38/71 (53.5%) hospitals. Most were Level 1 trauma centres (n = 23, 60.5%); 76% (n = 16) follow-up services were permanently funded. Follow-up services were led by a range of health professionals with over 60% (n = 19) identifying as trauma specialists. Patient inclusion criteria varied; only one service allowed self-referral (3.3%). Follow-up was within two weeks of acute care discharge in 53% (n = 16) of services. Care activities focused on physical health; psychosocial assessments were the least common. Most services provided care for adults and paediatric trauma (60.5%, n = 23); no service incorporated follow-up for family members. Evaluation of follow-up care was largely as part of a health service initiative; only three sites stated evaluation was specific to trauma follow-up.

CONCLUSION

Follow-up care is provided by trauma specialists and predominantly focuses on the physical health of the patients affected by major traumatic injury. Variations exist in terms of patient selection, reason for follow-up and care activities delivered with gaps in the provision of psychosocial and family health services identified. Currently, evaluation of trauma follow-up care is limited, indicating a need for further development to ensure that the care delivered is safe, effective and beneficial to patients, families and healthcare organisations.

Consensus quality indicators for monitoring multiple sclerosis

Multiple sclerosis (MS) as a chronic, degenerative autoimmune disease of the central nervous system has a longitudinal and heterogeneous course with increasing treatment options and risk profiles requiring constant monitoring of a growing number of parameters. Despite treatment guidelines, there is a lack of strategic and individualised monitoring pathways, including respective quality indicators (QIs). To address this, we systematically developed transparent, traceable, and measurable QIs for MS monitoring. Through literature review, expert discussions, and consensus-building, existing QIs were identified and refined. In a two-stage online Delphi process involving MS specialists (on average 53 years old and with 25 years of professional experience), the QIs were evaluated for content, clarity, and intelligibility, resulting in a set of 24 QIs and checklists to assess the quality of care. The final QIs provide a structured approach to document, monitor, and enhance the quality of care for people with MS across their treatment journey.

Implementation of an audit and feedback module targeting low-value clinical practices in a provincial trauma quality assurance program: a cost-effectiveness study

BACKGROUND

Audit and Feedback (A&F) interventions based on quality indicators have been shown to lead to significant improvements in compliance with evidence-based care including de-adoption of low-value practices (LVPs). Our primary aim was to evaluate the cost-effectiveness of adding a hypothetical A&F module targeting LVPs for trauma admissions to an existing quality assurance intervention targeting high-value care and risk-adjusted outcomes. A secondary aim was to assess how certain A&F characteristics might influence its cost-effectiveness.

METHODS

We conducted a cost-effectiveness analysis using a probabilistic static decision analytic model in the Québec trauma care continuum. We considered the Québec Ministry of Health perspective. Our economic evaluation compared a hypothetical scenario in which the A&F module targeting LVPs is implemented in a Canadian provincial trauma quality assurance program to a status quo scenario in which the A&F module is not implemented. In scenarios analyses we assessed the impact of A&F characteristics on its cost-effectiveness. Results are presented in terms of incremental costs per LVP avoided.

RESULTS

Results suggest that the implementation of A&F module (Cost = $1,480,850; Number of LVPs = 6,005) is associated with higher costs and higher effectiveness compared to status quo (Cost = $1,124,661; Number of LVPs = 8,228). The A&F module would cost $160 per LVP avoided compared to status quo. The A&F module becomes more cost-effective with the addition of facilitation visits; more frequent evaluation; and when only high-volume trauma centers are considered.

CONCLUSION

A&F module targeting LVPs is associated with higher costs and higher effectiveness than status quo and has the potential to be cost-effective if the decision-makers' willingness-to-pay is at least $160 per LVP avoided. This likely represents an underestimate of true ICER due to underestimated costs or missed opportunity costs. Results suggest that virtual facilitation visits, frequent evaluation, and implementing the module in high-volume centers can improve cost-effectiveness.

A multidisciplinary Delphi consensus to define evidence-based quality indicators for diabetic foot ulcer care

BACKGROUND

Valid measures to assess quality of care delivered to patients with diabetes suffering from diabetic foot ulcer (DFU) are scarce. This study aimed to achieve consensus on relevant and feasible quality indicators (QIs) among stakeholders involved in DFU care and was conducted as the second part of a Belgian QI selection study that sought to identify QIs for DFU care.

METHODS

A stakeholder panel, including caregivers from primary care and specialized disciplines active in diabetic foot care as well as a patient organization representative, was recruited. By using the RAND/UCLA Appropriateness Method, stakeholders were asked to rate a list of 42 candidate evidence-based indicators for appropriateness through a 9-point Likert scale. QIs were classified based on the median ratings and the disagreement index, calculated by the inter-percentile range adjusted for symmetry.

RESULTS

At the end of a three-phase process, 17 QIs were judged as appropriate. Among them, five were not previously described, covering the following topics: integration of wound care specialty in the multidisciplinary team, systematic evaluation of the nutritional status of the patient, administration of low-density lipoprotein-cholesterol lowering medication and protocolized care (implementation of care and prevention management protocols).

CONCLUSIONS

The identified evidence-based QIs provide an assessment tool to evaluate and monitor quality of care delivered to DFU patients. Future research should focus on their complementarity with the existing QIs and their implementation in clinical practice.

Developing an initial set of quality indicators for chiropractic care: a scoping review

BACKGROUND

Quality indicators are standardized, evidence-based measures of health care quality. Currently, there is no basic set of quality indicators for chiropractic care published in peer-reviewed literature. The goal of this research is to develop a preliminary set of quality indicators, measurable with administrative data.

METHODS

We conducted a scoping review searching PubMed/MEDLINE, CINAHL, and Index to Chiropractic Literature databases. Eligible articles were published after 2011, in English, developing/reporting best practices and clinical guidelines specifically developed for, or directly applicable to, chiropractic care. Eligible non-peer-reviewed sources such as quality measures published by the Centers for Medicare and Medicaid Services and the Royal College of Chiropractors quality standards were also included. Following a stepwise eligibility determination process, data abstraction identified specific statements from included sources that can conceivably be measured with administrative data. Once identified, statements were transformed into potential indicators by: 1) Generating a brief title and description; 2) Documenting a source; 3) Developing a metric; and 4) Assigning a Donabedian category (structure, process, outcome). Draft indicators then traversed a 5-step assessment: 1) Describes a narrowly defined structure, process, or outcome; 2) Quantitative data can conceivably be available; 3) Performance is achievable; 4) Metric is relevant; 5) Data are obtainable within reasonable time limits. Indicators meeting all criteria were included in the final set.

RESULTS

Literature searching revealed 2562 articles. After removing duplicates and conducting eligibility determination, 18 remained. Most were clinical guidelines (n = 10) and best practice recommendations (n = 6), with 1 consensus and 1 clinical standards development study. Data abstraction and transformation produced 204 draft quality indicators. Of those, 57 did not meet 1 or more assessment criteria. After removing duplicates, 70 distinct indicators remained. Most indicators matched the Donabedian category of process (n = 35), with 31 structure and 4 outcome indicators. No sources were identified to support indicator development from patient perspectives.

CONCLUSIONS

This article proposes a preliminary set of 70 quality indicators for chiropractic care, theoretically measurable with administrative data and largely obtained from electronic health records. Future research should assess feasibility, achieve stakeholder consensus, develop additional indicators including those considering patient perspectives, and study relationships with clinical outcomes.

TRIAL REGISTRATION

Open Science Framework, https://osf.io/t7kgm.

Nationwide consensus on quality indicators to assess glaucoma care: A modified Delphi approach

PURPOSE

Performance assessments are essential to tracking and improving quality in health care systems. Key aspects of the care process that act as indicators must be measured in order to gain an in-depth understanding of a care unit's operation. Without standardized quality indicators (QIs), it is difficult to characterize and compare the abilities of institutions to achieve excellence. The aim of this study is to reach a consensus among glaucoma specialists concerning the development of a set of QIs to assess the performance of glaucoma care units.

METHODS

A two-round Delphi technique was performed among glaucoma specialists in Portugal, using a 7-point Likert scale. Fifty-three initial statements (comprising process, structure, and outcome indicators) were evaluated and participants had to agree on which ones would be part of the final set of QIs.

RESULTS

By the end of both rounds, 28 glaucoma specialists reached consensus on 30/53 (57%) statements, including 19 (63%) process indicators (mainly relating to the proper implementation of complementary exams and the setting of follow-up intervals), 6 (20%) structure indicators, and 5 (17%) outcome indicators. Of the indicators that were part of the final list, functional and structural aspects of glaucoma progression and the availability of surgical/laser procedures were the most prevalent.

CONCLUSIONS

A set of 30 QIs for measuring the performance of glaucoma units was developed using a consensus methodology involving experts in the field. Their use as measurement standards would provide important information about unit operations and allow further implementation of quality improvements.

Proposed Quality Indicators for Aspects of Pediatric Acute Otitis Media Management

BACKGROUND

The high incidence of pediatric acute otitis media (AOM) makes the implications of overdiagnosis and overtreatment far-reaching. Quality indicators (QIs) for AOM are limited, drawing from generalized upper respiratory infection QIs, or locally developed benchmarks. Recognizing this, we sought to develop pediatric AOM QIs to build a foundation for future quality improvement efforts.

METHODS

Candidate indicators (CIs) were extracted from existing guidelines and position statements. The modified RAND Corporation/University of California, Los Angeles (RAND/UCLA) appropriateness methodology was used to select the final QIs by an 11-member expert panel consisting of otolaryngology-head and neck surgeons, a pediatrician and family physician.

RESULTS

Twenty-seven CIs were identified after literature review, with an additional CI developed by the expert panel. After the first round of evaluations, the panel agreed on 4 CIs as appropriate QIs. After an expert panel meeting and subsequent second round of evaluations, the panel agreed on 8 final QIs as appropriate measures of high-quality care. The 8 final QIs focus on topics of antimicrobial management, specialty referral, and tympanostomy tube counseling.

CONCLUSIONS

Evidence of variable and substandard care persists in the diagnosis and management of pediatric AOM despite the existence of high-quality guidelines. This study proposes 8 QIs which compliment guideline recommendations and are meant to facilitate future quality improvement initiatives that can improve patient outcomes.

Exploring stakeholders' perspectives on the quality of services provided through community pharmacies

INTRODUCTION

It is important to understand the factors impacting the quality of services provided through community pharmacies. Exploring how key stakeholders perceive good quality in these services is a logical first step. This could also inform the development of quality measures, such as quality indicators (QIs).

AIM

To identify key stakeholders' perspectives on the quality of services provided through community pharmacies in Norway, specifically by exploring their experiences and perceptions about what constitutes good service quality.

METHODS

A convenient sampling approach was used to recruit participants for five semi-structured focus groups from Facebook, pharmacy chains, and patient organizations. The interviews with twenty-six participants were conducted via Microsoft Teams. Interviews were transcribed verbatim, and an inductive thematic analysis with a reflexive approach was used.

RESULTS

Four main themes emerged from the analysis; 1) sufficient and substantively suitable information to cover individual needs, 2) communication skills and relationships with the pharmacy professionals, 3) customer satisfaction with knowledgeable employees and conveniently located pharmacies, and 4) factors that affect the pharmacy working environment.

CONCLUSION

This study has identified areas that pharmacy professionals and customers regard as essential to define good quality of community pharmacy services. Effective communication skills, appropriate provision of information, customer satisfaction, and working environment are all essential factors when developing quality metrics for community pharmacies.

Process, structural, and outcome quality indicators to support perioperative opioid stewardship: a rapid review

Opioids are effective analgesics but can cause harm. Opioid stewardship is key to ensuring that opioids are used effectively and safely. There is no agreed set of quality indicators relating to the use of opioids perioperatively. This work is part of the Yorkshire Cancer Research Bowel Cancer Quality Improvement programme and aims to develop useful quality indicators for the improvement of care and patient outcomes at all stages of the perioperative journey.A rapid review was performed to identify original research and reviews in which quality indicators for perioperative opioid use are described. A data tool was developed to enable reliable and reproducible extraction of opioid quality indicators.A review of 628 abstracts and 118 full-text publications was undertaken. Opioid quality indicators were identified from 47 full-text publications. In total, 128 structure, process and outcome quality indicators were extracted. Duplicates were merged, with the final extraction of 24 discrete indicators. These indicators are based on five topics: patient education, clinician education, pre-operative optimization, procedure, and patient-specific prescribing and de-prescribing and opioid-related adverse drug events.The quality indicators are presented as a toolkit to contribute to practical opioid stewardship. Process indicators were most commonly identified and contribute most to quality improvement. Fewer quality indicators relating to intraoperative and immediate recovery stages of the patient journey were identified. An expert clinician panel will be convened to agree which of the quality indicators identified will be most valuable in our region for the management of patients undergoing surgery for bowel cancer.

Evaluating the effectiveness of a multifaceted intervention to reduce low-value care in adults hospitalized following trauma: a protocol for a pragmatic cluster randomized controlled trial

BACKGROUND

While simple Audit & Feedback (A&F) has shown modest effectiveness in reducing low-value care, there is a knowledge gap on the effectiveness of multifaceted interventions to support de-implementation efforts. Given the need to make rapid decisions in a context of multiple diagnostic and therapeutic options, trauma is a high-risk setting for low-value care. Furthermore, trauma systems are a favorable setting for de-implementation interventions as they have quality improvement teams with medical leadership, routinely collected clinical data, and performance-linked to accreditation. We aim to evaluate the effectiveness of a multifaceted intervention for reducing low-value clinical practices in acute adult trauma care.

METHODS

We will conduct a pragmatic cluster randomized controlled trial (cRCT) embedded in a Canadian provincial quality assurance program. Level I-III trauma centers (n = 30) will be randomized (1:1) to receive simple A&F (control) or a multifaceted intervention (intervention). The intervention, developed using extensive background work and UK Medical Research Council guidelines, includes an A&F report, educational meetings, and facilitation visits. The primary outcome will be the use of low-value initial diagnostic imaging, assessed at the patient level using routinely collected trauma registry data. Secondary outcomes will be low-value specialist consultation, low-value repeat imaging after a patient transfer, unintended consequences, determinants for successful implementation, and incremental cost-effectiveness ratios.

DISCUSSION

On completion of the cRCT, if the intervention is effective and cost-effective, the multifaceted intervention will be integrated into trauma systems across Canada. Medium and long-term benefits may include a reduction in adverse events for patients and an increase in resource availability. The proposed intervention targets a problem identified by stakeholders, is based on extensive background work, was developed using a partnership approach, is low-cost, and is linked to accreditation. There will be no attrition, identification, or recruitment bias as the intervention is mandatory in line with trauma center designation requirements, and all outcomes will be assessed with routinely collected data. However, investigators cannot be blinded to group allocation and there is a possibility of contamination bias that will be minimized by conducting intervention refinement only with participants in the intervention arm.

TRIAL REGISTRATION

This protocol has been registered on ClinicalTrials.gov (February 24, 2023, # NCT05744154 ).

Identifying Quality Indicators for the Care of Hospitalized Injured Older Adults: A Scoping Review of the Literature

OBJECTIVES

Older adults represent more than 50% of trauma admissions in many high-income countries. Furthermore, they are at increased risk for complications, resulting in worse health outcomes than younger adults and a significant health care utilization burden. Quality indicators (QIs) are used to assess the quality of care in trauma systems, but few QIs reflect responses to older patients' specific needs. We aimed to (1) identify QIs used to assess acute hospital care for injured older patients, (2) assess support for identified QIs and, (3) identify gaps in existing QIs.

DESIGN

Scoping review of the scientific and gray literature.

METHODS

Selection and data extraction were performed by 2 independent reviewers. The level of support was assessed by the number of sources reporting QIs and whether they were developed according to scientific evidence, expert consensus, and patients' perspectives.

RESULTS

Of 10,855 identified studies, 167 were eligible. Among 257 different QIs identified, 52% were hip fracture specific. Gaps were identified for head injuries, rib, and pelvic ring fractures. Although 61% of QIs assessed care processes, 21% and 18% focused on structures and outcomes, respectively. Although most QIs were based on literature reviews and/or expert consensus, patients' perspective was rarely accounted for. The 15 QIs with the highest level of support included minimum time between emergency department arrival and ward admission, minimum time to surgery for fractures, assessment by a geriatrician, orthogeriatric review for hip fracture patients, delirium screening, prompt and appropriate analgesia, early mobilizations, and physiotherapy.

CONCLUSION AND IMPLICATIONS

Multiple QIs were identified, but their level of support was limited, and important gaps were identified. Future work should focus on achieving consensus for a set of QIs to assess the quality of trauma care to older adults. Such QIs could be used for quality improvement and ultimately improve outcomes for injured older adults.

The cost of inflammatory bowel disease in high-income settings: a Lancet Gastroenterology & Hepatology Commission

The cost of caring for patients with inflammatory bowel disease (IBD) continues to increase worldwide. The cause is not only a steady increase in the prevalence of Crohn's disease and ulcerative colitis in both developed and newly industrialised countries, but also the chronic nature of the diseases, the need for long-term, often expensive treatments, the use of more intensive disease monitoring strategies, and the effect of the diseases on economic productivity. This Commission draws together a wide range of expertise to discuss the current costs of IBD care, the drivers of increasing costs, and how to deliver affordable care for IBD in the future. The key conclusions are that (1) increases in health-care costs must be evaluated against improved disease management and reductions in indirect costs, and (2) that overarching systems for data interoperability, registries, and big data approaches must be established for continuous assessment of effectiveness, costs, and the cost-effectiveness of care. International collaborations should be sought out to evaluate novel models of care (eg, value-based health care, including integrated health care, and participatory health-care models), as well as to improve the education and training of clinicians, patients, and policy makers.

Quality indicators in primary elective total hip and knee arthroplasty

PURPOSE

Total hip arthroplasty (THA) and total knee arthroplasty (TKA) represent a significant portion of healthcare spending and are high-priority for quality improvement initiatives. This study aims to develop quality indicators (QIs) in the care of primary elective THA and TKA patients. These QIs serve a number of purposes including documentation of the quality of care, objective comparisons of institutions/providers, facilitating pay-for-performance initiatives, and supporting accountability, regulation, and accreditation.

METHODS

A guideline-based approach, initially described by Kötter et al., was utilized. Eight clinical practice guidelines (CPGs) were evaluated for candidate indicators (CIs). CIs with high-quality evidence and consensus statements were extracted. Eighteen additional CIs were included from previous work that evaluated quality improvement databases. Each CI and supporting evidence was submitted for independent review by an expert panel. The RAND Corporation-University of California, Los Angeles (RAND/UCLA) appropriateness methodology was utilized and items were rated based on validity, reliability, and feasibility of measurement. After two rounds of ratings and ranking, a final ranked list of QIs was obtained.

RESULTS

Fifty-six CIs were identified from the literature and CPGs or proposed by the expert panel. Two rounds of voting resulted in 12 total QIs that were deemed appropriate measures of high-quality care. The final 12 QIs were ranked by order of importance: use of peri-operative tranexamic acid, infusion of prophylactic antibiotics prior to inflation of tourniquet, appropriate post-operative venous thromboembolic prophylaxis, complication rate, rate of secondary procedure, readmission rate, early mobilization, average change of pre- to one year post-operative functional status, use of multimodal analgesia, use of neuraxial anesthesia, use of peri-articular injection in TKA, and use of pre-operative PO analgesia.

CONCLUSION

This study is an expert opinion based on parameters observed in modern and high-quality academic settings. Twelve QIs are proposed to assess the quality of care in the peri-operative management of primary elective THA and TKA patients.

Identifying candidate quality indicators of tools that support the practice of knowledge translation: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to identify and characterize relevant knowledge translation methods tools (those that provide guidance for optimized knowledge translation practice) to uncover candidate quality indicators to inform a future quality assessment tool for knowledge translation strategies.

INTRODUCTION

Knowledge translation strategies (defined as including knowledge translation interventions, tools, and products) target various knowledge users, including patients, clinicians, researchers, and policy-makers. The development and use of strategies that support knowledge translation practice have been rapidly increasing, making it difficult for knowledge users to decide which to use. There is limited evidence-based guidance or measures to help assess the overall quality of knowledge translation strategies.

INCLUSION CRITERIA

Empirical and non-empirical documents will be considered if they explicitly describe a knowledge translation methods tool and its development, evaluation or validation, methodological strengths or limitations, and/or use over time. The review will consider a knowledge translation methods tool if it falls within at least one knowledge translation domain (ie, implementation, dissemination, sustainability, scalability, integrated knowledge translation) in the health field.

METHODS

We will conduct a systematic search of relevant electronic databases and gray literature. The search strategy will be developed iteratively by an experienced medical information specialist and peer-reviewed with the PRESS checklist. The search will be limited to English-only documents published from 2005 onward. Documents will be independently screened, selected, and extracted by 2 researchers. Data will be analyzed and summarized descriptively, including the characteristics of the included documents, knowledge translation methods tools, and candidate quality indicators.

SCOPING REVIEW REGISTRATION

Open Science Framework ( https://osf.io/chxvq ).

Development and validation of a new set of indicators to assess the quality of maternal and child nutritional care at the primary care

Introduction

Maternal and child malnutrition is a worldwide public health problem with short, medium, and long-term adverse consequences for both mother and child. In Mexico, maternal and child malnutrition represents a serious public health problem that must be urgently addressed. In this context, Primary Health Care (PHC) plays an important role in the prevention, detection, monitoring, and treatment of the different forms of maternal and child malnutrition. Assessing the quality of nutritional care offered at this level of care is necessary in order to improve it; however, there are no indicators for the evaluation of this quality. Therefore, this study aimed at developing a set of indicators to assess the quality of maternal and child nutritional care at PHC.

Methods

We developed indicators for different stages of life: preconception, pregnancy, infancy, and preschool age. A systematic review of the literature on clinical guidelines for the prevention, diagnosis, and treatment of the different forms of malnutrition was carried out; the recommendations of the guidelines evaluated with good quality were extracted.

Results

Based on these recommendations, 22 indicators were constructed. A pilot study was carried out to validate the indicators and 16 indicators were selected to assess the maternal and child nutritional care at PHC.

Global core indicators for measuring WHO's paediatric quality-of-care standards in health facilities: development and expert consensus

BACKGROUND

There are currently no global recommendations on a parsimonious and robust set of indicators that can be measured routinely or periodically to monitor quality of hospital care for children and young adolescents. We describe a systematic methodology used to prioritize and define a core set of such indicators and their metadata for progress tracking, accountability, learning and improvement, at facility, (sub) national, national, and global levels.

METHODS

We used a deductive methodology which involved the use of the World Health Organization Standards for improving the quality-of-care for children and young adolescents in health facilities as the organizing framework for indicator development. The entire process involved 9 complementary steps which included: a rapid literature review of available evidence, the application of a peer-reviewed systematic algorithm for indicator systematization and prioritization, and multiple iterative expert consultations to establish consensus on the proposed indicators and their metadata.

RESULTS

We derived a robust set of 25 core indicators and their metadata, representing all 8 World Health Organization quality standards, 40 quality statements and 520 quality measures. Most of these indicators are process-related (64%) and 20% are outcome/impact indicators. A large proportion (84%) of indicators were proposed for measurement at both outpatient and inpatient levels. By virtue of being a parsimonious set and given the stringent criteria for prioritizing indicators with "quality measurement" attributes, the recommended set is not evenly distributed across the 8 quality standards.

CONCLUSIONS

To support ongoing global and national initiatives around paediatric quality-of-care programming at country level, the recommended indicators can be adopted using a tiered approach that considers indicator measurability in the short-, medium-, and long-terms, within the context of the country's health information system readiness and maturity. However, there is a need for further research to assess the feasibility of implementing these indicators across contexts, and the need for their validation for global common reporting.

Putting measurement on a diet: development of a core set of indicators for quality improvement in the ICU using a Delphi method

Background

The number and efficacy of indicators used to monitor and improve the quality of care in Intensive Care Units (ICU) is debatable. This study aimed to select a consensus-based core set of indicators for effective quality improvement in the ICU.

Methods

A Delphi study with a panel of intensivists, ICU nurses, and former ICU patients or relatives (n = 34) from general, teaching, and academic hospitals. Panelists completed a questionnaire in which they scored 69 preselected quality indicators on relevance using a nine-point Likert scale. Indicators were categorized using the rated relevance score into: ‘accepted, ‘equivocal’ and ‘excluded’. Questionnaire results were discussed in focus groups to reach consensus on the final set.

Results

Response rates for the questionnaire and focus groups were 100 and 68%, respectively. Consensus was reached on a final set of 17 quality indicators including patient reported outcome measures (PROMs) and patient reported experience measures (PREMs). Other quality indicators relate to the organization and outcome of ICU care, including safety culture, ICU standardized mortality ratio, and the process indicator ‘learning from and improving after serious incidents’.

Conclusions

ICU clinicians and former patients and relatives developed a consensus-based core set of ICU quality indicators that is relatively short but comprehensive and particularly tailored to end-users needs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08236-3.

Examining Interactions Across Instructional Tiers: Do Features of Tier 1 Predict Student Responsiveness to Tier 2 Mathematics Intervention?

High-quality Tier 1 instruction is frequently conceptualized as the "foundation" for other tiers of intervention within multitiered systems of support (MTSS) models. However, the vast majority of Tier 2 intervention studies do not account for Tier 1 variables when examining intervention effectiveness. The purpose of this study was to examine Tier 1 predictors, or "quality indicators," of differential responsiveness to Tier 2 mathematics intervention. Data were drawn from a large-scale data set where all teachers taught the Early Learning in Mathematics (Tier 1) core program across the academic year, and a subset of students were selected for the ROOTS (Tier 2) mathematics intervention. We examined the following Tier 1 variables: (a) classroom-level mathematics gains, (b) Tier 1 fidelity of implementation, (c) Tier 1 classroom management and instructional support, and (d) class size. Response to Tier 2 intervention was not significantly predicted by any of the Tier 1 variables examined; however, the pattern of Hedges' g effect sizes suggested that students with higher quality of Tier 1 instruction tended to benefit less from the Tier 2 ROOTS intervention. Results are discussed in the context of implications for research and practice.

The role of implementation science in improving epilepsy surgery utilization

Randomized controlled studies demonstrated that patients with intractable epilepsy could benefit significantly more from epilepsy surgery than from continuing medical therapy. Unfortunately, robust efforts over the last few decades, including the formation and dissemination of guidelines and practice parameters, did not improve the utilization of epilepsy surgery. Epilepsy surgery remains one of the most underutilized evidence-based interventions in modern medicine. A new scientific study of methods has emerged to improve uptake of evidence-based practices, named implementation science (IS). Despite its tremendous rise in popularity in various domains, its usage to mitigate epilepsy surgery underutilization is very limited. In fact, the application of principles and methods of IS are somewhat restricted in the entire neuroscience field, where quality improvement (QI) efforts primarily drive the provision of high-quality health care. Although both QI efforts and IS have a similar goal of improving healthcare quality, they differ significantly in associated terminologies, concepts, and approaches. For implementing high-quality, evidence-based practices in routine clinical settings, we need a better understanding of IS methods and closer integration between QI and IS fields. Recognizing a dearth of awareness of IS in the neuroscience community, the first part of the review addresses the fundamentals of IS, focusing on multifaceted implementation strategies that neurologists can apply in their clinical practice. In the second part of the review, an entire illustrative case is presented to familiarize neurologists with the practical application of diverse implementation strategies to mitigate the underutilization of epilepsy surgery.

Quality Indicators Targeting Low-Value Clinical Practices in Trauma Care

Importance

The use of quality indicators has been shown to improve injury care processes and outcomes. However, trauma quality indicators proposed to date exclusively target the underuse of recommended practices. Initiatives such as Choosing Wisely publish lists of practices to be questioned, but few apply to trauma care, and most have not successfully been translated to quality indicators.

Objective

To develop a set of evidence and patient-informed, consensus-based quality indicators targeting reductions in low-value clinical practices in acute, in-hospital trauma care.

Design, Setting, and Participants

This 2-round Research and Development/University of California at Los Angeles (RAND/UCLA) consensus study, conducted from April 20 to June 9, 2021, comprised an online questionnaire and a virtual workshop led by 2 independent moderators. Two panels of international experts from Canada, Australia, the US, and the UK, and local stakeholders from Québec, Canada, represented key clinical expertise involved in trauma care and included 3 patient partners.

Main Outcomes and Measures

Panelists were asked to rate 50 practices on a 7-point Likert scale according to 4 quality indicator criteria: importance, supporting evidence, actionability, and measurability.

Results

Of 49 eligible experts approached, 46 (94%; 18 experts [39%] aged ≥50 years; 37 men [80%]) completed at least 1 round and 36 (73%) completed both rounds. Eleven quality indicators were selected overall, 2 more were selected by the international panel and a further 3 by the local stakeholder panel. Selected indicators targeted low-value clinical practices in the following aspects of trauma care: (1) initial diagnostic imaging (head, cervical spine, ankle, and pelvis), (2) repeated diagnostic imaging (posttransfer computed tomography [CT] and repeated head CT), (3) consultation (neurosurgical and spine), (4) surgery (penetrating neck injury), (5) blood product administration, (6) medication (antibiotic prophylaxis and late seizure prophylaxis), (7) trauma service admission (blunt abdominal trauma), (8) intensive care unit admission (mild complicated traumatic brain injury), and (9) routine blood work (minor orthopedic surgery).

Conclusions and Relevance

In this consensus study, a set of consensus-based quality indicators were developed that were informed by the best available evidence and patient priorities, targeting low-value trauma care. Selected indicators represented a trauma-specific list of practices, the use of which should be questioned. Trauma quality programs in high-income countries may use these study results as a basis to select context-specific quality indicators to measure and reduce low-value care.

Associations between quality of health care and clinical outcomes in patients with rheumatic and musculoskeletal diseases: a rehabilitation cohort study

BACKGROUND

The quality of provided health care may be an important source of variation in rehabilitation outcomes, increasing the interest in associations between quality indicators (QIs) and improved patient outcomes. Therefore, we examined the associations between the quality of rehabilitation processes and subsequent clinical outcomes among patients with rheumatic and musculoskeletal diseases (RMDs).

METHODS

In this multicentre prospective cohort study, adults with RMDs undergoing multidisciplinary rehabilitation at eight participating centres reported the quality of rehabilitation after 2 months and outcomes after 2, 7, and 12 months. We measured perceived quality of rehabilitation by 11 process indicators that cover the domains of initial assessments, patient participation and individual goal-setting, and individual follow-up and coordination across levels of health care. The patients responded "yes" or "no" to each indicator. Scores were calculated as pass rates (PRs) from 0 to 100% (best score). Clinical outcomes were goal attainment (Patient-Specific Functional Scale), physical function (30 s sit-to-stand test), and health-related quality of life (EuroQoL 5D-5L). Associations between patient-reported quality of care and each outcome measure at 7 months was analysed by linear mixed models.

RESULTS

A total of 293 patients were enrolled in this study (mean age 52 years, 76% female). Primary diagnoses were inflammatory rheumatic disease (64%), fibromyalgia syndrome (18%), unspecific neck, shoulder, or low back pain (8%), connective tissue disease (6%), and osteoarthritis (4%). The overall median PR for the process indicators was 73% (range 11-100%). The PR was lowest (median 40%) for individual follow-up and coordination across levels of care. The mixed model analyses showed that higher PRs for the process indicators were not associated with improved goal attainment or improved physical function or improved health-related quality of life.

CONCLUSIONS

The quality of rehabilitation processes was not associated with important clinical outcomes. An implication of this is that measuring only the outcome dimension of quality may result in incomplete evaluation and monitoring of the quality of care, and we suggest using information from both the structure, process, and outcome dimensions to draw inferences about the quality, and plan future quality initiatives in the field of complex rehabilitation.

TRIAL REGISTRATION

The study is part of the larger BRIDGE trial (ClinicalTrials.gov NCT03102814 ).

Purpose, Subject, and Consumer Comment on "Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study"

Zegers and colleagues' study codifies the perceived burden of quality monitoring and improvement stemming from the work by clinicians of registering (documenting) quality information in the medical record. We agree with Zegers and colleagues' recommendation that a smaller, more effective and curated set of measures is needed to reduce burden, confusion, and expense. We further note that focusing on validity of clinical evidence behind individual measures is critical, but insufficient. We therefore extend Zegers and colleagues' work through a pragmatic, tripartite heuristic. To assess the value of and curate a targeted set of performance measures, we propose concentrating on the relationships among three factors: (1) The purpose of the performance measure, (2) the subject being evaluated, and (3) the consumer using information for decision-making. Our proposed tripartite framework lays the groundwork for executing the evidence-based recommendations proposed by Zegers et al, and provides a path forward for more effective healthcare performance-measurement systems.

Factors influencing quality nutrition service provision at antenatal care contacts: Findings from a public health facility-based observational study in 21 districts of Bangladesh

Malnutrition during pregnancy is associated with increased maternal morbidity and mortality and has a long-term negative impact on child growth and development. Antenatal care (ANC) is the formal point of contact for pregnant women to receive preventive health and nutrition services. We assessed the quality of nutrition service delivery during ANC and examined its influencing factors related to the health facility, health care provider (HCP) and client characteristics. We conducted a cross-sectional assessment in 179 facilities, including 1,242 ANC observations and exit interviews of pregnant women from 21 districts in Bangladesh. We considered four essential nutrition services at each ANC contact including maternal weight measurement, anaemia assessment, nutrition counselling and iron-folic acid (IFA) supplement provision. We defined a composite ‘quality nutrition service’ outcome by counting the number of services (out of four) provided at each ANC from observation data. We explored both the supply-side and the client-level factors of quality nutrition service using multilevel Poisson regression. Overall, only 15% of clients received all four nutrition services. Performance of weight measurement (79%) was higher than IFA provision (56%), anaemia assessment (52%) and nutrition counselling (52%). The multivariable analysis showed that quality nutrition service delivery is positively associated with good logistical readiness of the facilities (aIRR: 1.23, 95% CI: 1.08–1.39), consultation by paramedics (aIRR 1.23, 95% CI: 1.06–1.42) and community health care providers (aIRR 1.32, 95% CI: 1.12–1.57), HCPs’ knowledge on maternal nutrition (aIRR 1.04; 95% CI: 1.01–1.08), better HCP-client communication (aIRR 1.14; 95% CI: 1.04–1.26) and use visual aids or ANC card (aIRR 1.18; 95% CI: 1.11–1.27). We found limited associations between HCP training and external supervision with the quality of nutrition services. In conclusion, the quality of nutrition service provision during ANC is suboptimal. Public health nutrition programmers should ensure the facilities’ logistical readiness, and revisit and reinforce the content and modality of training and supportive supervision of the HCPs. They should also emphasize positive HCP-client communication and the use of job aids to improve the quality of nutrition service provision during ANC.

Quality of Care Indicators for Hospital Physical Therapy Units: A Systematic Review

OBJECTIVE

The purpose of this review was to identify quality indicators described in the literature that may be used as quality measures in hospital physical therapy units.

METHODS

The following sources were searched for quality indicators or articles: Web of Science, MEDLINE, IBECS, Latin American and Caribbean Health Sciences Literature, Cumulative Index of Nursing and Allied Health, Academic Search Complete, SportDiscus, SciELO, PsychINFO, Consejo Superior de Investigaciones Cientificas, and Scopus databases; the Agency for Healthcare Research and Quality, National Health System Indicator Portal, Joint Commission on Accreditation of Healthcare Organizations, and Organisation for Economic Co-operation and Development websites; and the National Quality Forum's measures inventory tool. Search terms included "quality indicator," "quality measure," "physiotherapy," and "physical therapy." Inclusion criteria were articles written in English, Spanish, French, or Portuguese aimed at measuring the quality of care in hospital physical therapy units. Evidence-based indicators with an explicit formula were extracted by 2 independent reviewers and then classified using the structure-process-outcome model, quality domain, and categories defined by a consensus method.

RESULTS

Of the 176 articles identified, only 19 met the criteria. From these articles and from the indicator repository searches, 178 clinical care indicators were included in the qualitative synthesis and presented in this paper. Process and outcome measures were prevalent, and 5 out of the 6 quality domains were represented. No efficiency measures were identified. Moreover, structure indicators, equity and accessibility indicators, and indicators in the cardiovascular and circulatory, mental health, pediatrics, and intensive care categories were underrepresented.

CONCLUSIONS

A broad selection of quality indicators was identified from international resources, which can be used to measure the quality of physical therapy care in hospital units.

IMPACT

This review identified 178 quality of care indicators that can be used in clinical practice monitoring and quality improvement of hospital physical therapy units. The results highlight a lack of accessibility, equity, and efficiency measures for physical therapy units.

Quality Indicators for the Diagnosis and Management of Primary Hyperparathyroidism

Importance

Primary hyperparathyroidism (pHPT) is a common endocrine disorder with many diagnostic and treatment challenges. Despite high-quality guidelines, care is variable, and there is low adherence to evidence-based treatment pathways.

Objective

To develop quality indicators (QIs) to evaluate the diagnosis and treatment of pHPT that could measure, improve, and optimize quality of care and outcomes for patients with this disease.

Design, Setting, and Participants

This quality improvement study used a guideline-based approach to develop QIs that were ranked by a Canadian 9-member expert panel of 3 endocrinologists, 3 otolaryngologists, and 3 endocrine surgeons. Data were analyzed between September 2020 and May 2021.

Main Outcomes and Measures

Candidate indicators (CIs) were extracted from published primary hyperparathyroidism guidelines and summarized with supporting evidence. The 9-member expert panel rated each CI on the validity, reliability, and feasibility of measurement. Final QIs were selected from CIs using the modified RAND-University of California, Los Angeles appropriateness methodology. All panelists were then asked to rank the top 5 QIs for primary, endocrine, and surgical care.

Results

Forty QIs were identified and evaluated by the expert panel. After 2 rounds of evaluations and discussion, a total of 18 QIs were selected as appropriate measures of high-quality care. The top 5 QIs for primary, endocrine, and surgical care were selected following panelist rankings.

Conclusions and Relevance

This quality improvement study proposes 18 QIs for the diagnosis and management of pHPT. Furthermore, the top 5 QIs applicable to physicians commonly treating pHPT, including general physicians, internists, endocrinologists, otolaryngologists, and surgeons, are included. These QIs not only assess the quality of care to guide the process of improvement, but also can assess the implementation of evidence-based guideline recommendations. Using these indicators in clinical practice and health system registries can improve quality and cost-effectiveness of care for patients with pHPT.

Implementation Science to Improve Quality of Neurological Care

Neurological disorders are the leading cause of disability and the second leading cause of death globally. To challenge this enormous disease burden, scientists are pursuing innovative solutions to maintain and improve the quality of neurological care. Despite the availability of many effective evidence-based practices, many patients with neurological disorders cannot access these (or receive them inefficiently after a long delay) and may be exposed to unnecessary, expensive, and potentially harmful treatments. To promote the systematic uptake of evidence-based practices into the real world, a new scientific study of methods has been developed: implementation science. In implementation science research, transdisciplinary research teams systematically (using theory, model, and framework) assess local barriers to facilitate the adoption of evidence-based practices and examine potential solutions using implementation strategies (interventions that help adoption of intended practices) targeting multiple levels in the health care system, including patient, provider, clinic, facility, organization, or broader community and policy environment. The success of these strategies (implementation outcomes) is measured by the extent and quality of the implementation. Implementation studies can be either observational or interventional but are distinct from traditional efficacy or effectiveness studies. Traditional neuroscience research and clinical trials, conducted in controlled settings, focus on discovering new insights with little consideration of translating those insights into the everyday practice of a resource-constrained and dynamic health care system. Thus, neurologists should become familiar with implementation science to reduce the knowledge-practice gap, maximize health care value, and improve management of brain disorders affecting public health.

An Environmental Scan and Evaluation of Quality Indicators Across Canadian Kidney Transplant Centers

Background:

Kidney transplantation is the optimal treatment for an individual requiring kidney replacement therapy, resulting in improved survival and quality of life while costing the health care system less than maintenance dialysis. Achieving and maintaining a kidney transplant requires extensive coordination of several different health care services. To improve the quality of kidney transplant care, quality metrics or indicators that encompass all aspects of the individual’s journey to transplant should be measured in a standardized fashion.

Objective:

To identify, categorize, and evaluate strengths and weaknesses of kidney transplant quality indicators currently being used across Canada.

Design:

An environmental scan of quality indicators being used by kidney organizations and programs.

Setting:

A 16-member volunteer pan-Canadian panel with expertise in nephrology, transplant, and quality improvement.

Sample:

Transplant programs, as well as provincial transplant and kidney agencies across Canada.

Methods:

Indicators were first categorized based on the period of transplant care and then using the Institute of Medicine and Donabedian frameworks. A 4-member subcommittee rated each indicator using a modified version of the Delphi consensus technique based on the American College of Physician/Agency for Healthcare Research and Quality criteria. Consensus ratings were subsequently shared with the entire 16-member panel for additional comments.

Results:

We identified 46 measures related to transplant care across 7 Canadian provinces (9 referral and evaluation, 9 waitlist activity and outcomes, 6 hospitalization for transplant surgery, 12 posttransplant care, 6 organ utilization, 4 living donor). We rated 24 indicators (52%) as necessary to distinguish high-quality from low-quality care, most of which measured effective (n = 10) or efficient (n = 6) care. Only 7 (15%) of 46 indicators evaluated person-centered or equitable care. Fourteen common indicators were measured by 5 of 7 provinces, 10 of which were deemed “necessary,” measuring safe (n = 2), effective (n = 5), efficient (n = 2), and equitable (n = 1) care.

Limitations:

The panel lacked patient and allied health representation.

Conclusions:

There are a large number of kidney transplant quality indicators currently being used in Canada, some of which are common across provinces and focus primarily on measuring effective care. Person-centered and equitable care indicators were lacking, and only half of these indicators were deemed “necessary” for quality improvement. Our results should complement ongoing work to achieve national consensus on the standardization of quality indicators in kidney transplantation.

Quality Indicators for the Diagnosis and Management of Menière's Disease

OBJECTIVE

Menière's disease (MD) is a clinical disorder that often provides challenges in diagnosis and management. High-quality evidence to guide care providers is sparse, which can result in significant practice variations. Quality indicators (QIs) are one method that can be used to standardize and measure accepted care practices to improve healthcare quality and patient outcomes. Here, we developed practical, high-yield QIs that serve to measure and inform the quality of care provided to patients with MD.

STUDY DESIGN

Modified RAND Corporation University of California, Los Angeles appropriateness methodology for QI development.

SETTING

Multicenter nine-member expert panel.

PATIENTS

NA.

INTERVENTIONS

NA.

MAIN OUTCOME MEASURE

Final QIs deemed appropriate measures of quality care with agreement by the expert panel.

RESULTS

Twenty-seven candidate indicators were identified after literature review. After the first round of evaluations, the panel agreed on three candidate indicators as appropriate QIs. A subsequent expert panel meeting provided a platform to discuss disagreements. Two agreed-upon QIs were revised during this discussion before final evaluations. The expert panel ultimately agreed upon five QIs as appropriate measures of high-quality care after completing final evaluations and reviewing updated literature. The five quality indicators measure audiometric documentation, minimization of electrocochleography, use of intratympanic dexamethasone, use of intratympanic gentamycin, and rate of labyrinthectomy/vestibular neurectomy in refractory MD patient.

CONCLUSIONS

This study proposes five QIs that cover key aspects of care for MD, such as accurate diagnosis and management options including initial destructive therapies. These QIs can serve multiple purposes, the most important of which is to galvanize quality improvement initiatives.

Quality Indicators for the Diagnosis and Management of Sudden Sensorineural Hearing Loss

OBJECTIVE

Sudden sensorineural hearing loss (SSNHL) is an ideal entity for quality indicator (QI) development, providing treatment challenges resulting in variable or substandard care. The American Academy of Otolaryngology-Head and Neck Surgery recently updated their SSNHL guidelines. With SSNHL demonstrating a large burden of illness, this study sought to leverage the updated guidelines and develop QIs that support quality improvement initiatives at an individual, institutional, and systems level.

METHODS

Candidate indicators (CIs) were extracted from high-quality SSNHL guidelines that were evaluated using the Appraisal of Guidelines for Research and Evaluation II tool. Each CI and its supporting evidence were summarized and reviewed by a nine-member expert panel based on validity, reliability, and feasibility of measurement. Final QIs were selected from CIs using the modified RAND Corporation-University of California, Los Angeles appropriateness methodology.

RESULTS

Fifteen CIs were identified after literature review. After the first round of evaluations, the panel agreed on 11 candidate indicators as appropriate QIs with 2 additional CIs suggested for consideration. An expert panel meeting provided a platform to discuss areas of disagreement before final evaluations. The expert panel subsequently agreed upon 11 final QIs as appropriate measures of high-quality care for SSNHL.

CONCLUSION

The 11 proposed QIs from this study are supported by evidence and expert consensus, facilitating measurement across a wide breadth of quality domains. With the recently updated SSNHL guidelines, and a greater focus on quality improvement opportunities, these QIs may be used by healthcare providers for targeted quality improvement initiatives.

Derivation and validation of actionable quality indicators targeting reductions in complications for injury admissions

PURPOSE

Approximately, one out of five patients hospitalized following injury will develop at least one hospital complication, more than three times that observed for general admissions. We currently lack actionable Quality Indicators (QI) targeting specific complications in this population. We aimed to derive and validate QI targeting hospital complications for injury admissions and develop algorithms to identify patient charts to review.

METHODS

We conducted a retrospective cohort study including patients with major trauma admitted to any level I or II adult trauma center an integrated Canadian trauma system (2014-2019). We used the trauma registry to develop five QI targeting deep vein thrombosis/pulmonary embolism (DVT/PE), decubitus ulcers, delirium, pneumonia and urinary tract infection (UTI). We developed algorithms to identify patient charts to revise on consultation with a group of clinical experts.

RESULTS

The study population included 14,592 patients of whom 5.3% developed DVT or PE, 2.7% developed a decubitus ulcer, 8.6% developed delirium, 14.7% developed pneumonia and 7.3% developed UTI. The indicators demonstrated excellent predictive performance (Area Under the Curve 0.81-0.87). We identified 4 hospitals with a higher than average incidence of at least one of the targeted complications. The algorithms identified on average 50 and 20 charts to be reviewed per year for level I and II centers, respectively.

CONCLUSION

In line with initiatives to improve the quality of trauma care, we propose QI targeting reductions in hospital complications for injury admissions and algorithms to generate case lists to facilitate the review of patient charts.

Measuring Processes of Integrated Care for Hospital to Home Transitions

Background:

Integrated care is a promising approach to improve transitions from hospital for older adults. Measures of integrated care tend to be survey-based or outcomes focused. This study determined the feasibility of using hospital chart data to measure integrated processes of care.

Methods:

This paper reports on two objectives: 1) the development of an integrated care transition framework and associated features of care; 2) a pilot study to test if the features could be applied to 214 hospital patient charts.

Results:

Twenty-four features were tested, and fifteen features could be reliably measured using chart review. Of these, the percent of patients classified as receiving integrated care varied widely across the items, from 0.05% to 84.1%.

Discussion:

The framework presented in this paper can guide measurement of system and clinical delivery of integrated care transitions. In combination with other tools, chart review can provide perspective on day-to-day care delivery not otherwise accessible, and highlight areas requiring practice change.

Conclusion:

Multiple measurement perspectives are needed to improve our understanding of how integrated care is being implemented. While chart review cannot address the full breadth of integrated care, it can help understand how processes of care are being implemented in routine daily care.

Strategies for developing and implementing a rheumatoid arthritis healthcare quality framework: a thematic analysis of perspectives from arthritis stakeholders

OBJECTIVES

To obtain stakeholder perspectives to inform the development and implementation of a rheumatoid arthritis (RA) healthcare quality measurement framework.

DESIGN

Qualitative study using thematic analysis of focus groups and interviews.

SETTING

Arthritis stakeholders from across Canada including healthcare providers, persons living with RA, clinic managers and policy leaders were recruited for the focus groups and interviews.

PARTICIPANTS

Fifty-four stakeholders from nine provinces.

INTERVENTIONS

Qualitative researchers led each focus group/interview using a semistructured guide; the digitally recorded data were transcribed verbatim. Two teams of two coders independently analysed the transcripts using thematic analysis.

RESULTS

Perspectives on the use of different types of measurement frameworks in healthcare were obtained. In particular, stakeholders advocated for the use of existing healthcare frameworks over frameworks developed in the business world and adapted for healthcare. Persons living with RA were less familiar with specific measurement frameworks, however, they had used existing online public forums for rating their experience and quality of healthcare provided. They viewed a standardised framework as potentially useful for assisting with monitoring the care provided to them individually. Nine guiding principles for framework development and 13 measurement themes were identified. Perceived barriers identified included access to data and concerns about how measures in the framework were developed and used. Effective approaches to framework implementation included having sound knowledge translation strategies and involving stakeholders throughout the measurement development and reporting process. Clinical models of care and health policies conducive to outcome measurement were highlighted as drivers of successful measurement initiatives.

CONCLUSION

These important perspectives will be used to inform a healthcare quality measurement framework for RA.

Quality Indicators (QI) of Acute Pain after Surgery in European Countries

OBJECTIVES

It is common practice to use pain quality indicators (QI) to accurately assess the medical care situation of patients. This literature review identifies the status quo of acute pain QI among adult inpatients.

DATA SOURCES

Studies published in English or German were identified using a systematic search on CINAHL, Cochrane Library, PubMed, Web of Science, and Google Scholar from 01/2007 to 02/2018. Additional pain management journals, conference proceedings, and websites of health organizations and pain societies were manually screened. Studies about postoperative pain in adults (≥18 years) during inpatient stay after all types of surgery in Europe were included in this review.

REVIEW/ANALYSIS METHODS

The identified study results were categorized into structural, process, and outcome indicators based on Donabedian's framework of evaluating care quality.

RESULTS

The search identified 319 citations, of which 20 studies used structure, process, and outcome data including 180,988 patients and 1,970 health care professionals to gain insight into the quality of acute pain management. Overall, 80% used patient surveys to collect data. National data on pain management are reported in five European studies (France [2], the Netherlands, Spain, and Austria).

CONCLUSIONS

European studies comprehensively comparing acute pain management results are currently missing. Thus, this report highlights the need to develop consensus-based quality indicators in management of acute pain, which take into account both the methodologic quality and the relevance to clinical practice.

A quality indicator set for rehabilitation services for people with rheumatic and musculoskeletal diseases demonstrates adequate responsiveness in a pre-post evaluation

BACKGROUND

Quality of care is gaining increasing attention in research, clinical practice, and health care planning. Methods for quality assessment and monitoring, such as quality indicators (QIs), are needed to ensure health services in line with norms and recommendations. The aim of this study was to assess the responsiveness of a newly developed QI set for rehabiliation for people with rheumatic and musculoskeletal diseases (RMDs).

METHODS

We used two yes/no questionnaires to measure quality from both the provider and patient perspectives, scored in a range of 0-100% (best score, 100%). We collected QI data from a multicenter stepped-wedge cluster-randomized controlled trial (the BRIDGE trial) that compared traditional rehabilitation with a new BRIDGE program designed to improve quality and continuity in rehabilitation. Assessment of the responsiveness was performed as a pre-post evaluation: Providers at rehabilitation centers in Norway completed the center-reported QIs (n = 19 structure indicators) before (T1) and 6-8 weeks after (T2) adding the BRIDGE intervention. The patient-reported QIs comprised 14 process and outcomes indicators, measuring quality in health services from the patient perspective. Pre-intervention patient-reported data were collected from patients participating in the traditional program (T1), and post-intervention data were collected from patients participating in the BRIDGE program (T2). The patient groups were comparable. We used a construct approach, with a priori hypotheses regarding the expected direction and magnitude of PR changes between T1 and T2. For acceptable responsivess, at least 75% of the hypotheses needed to be confirmed.

RESULTS

All eight participating centers and 82% of the patients (293/357) completed the QI questionnaires. Responsiveness was acceptable, with 44 of 53 hypotheses (83%) confirmed for single indicators and 3 of 4 hypotheses (75%) confirmed for the sum scores.

CONCLUSION

We found this QI set for rehabilitation to be responsive when applied in rehabilitation services for adults with various RMD conditions. We recommend this QI set as a timely method for establishing quality-of-rehabilitation benchmarks, promoting important progress toward high-quality rehabilitation, and tracking trends over time.

TRIAL REGISTRATION

The study is part of the larger BRIDGE trial, registered at ClinicalTrials.gov (Identifier: NCT03102814).

Identification and operationalisation of indicators to monitor successful uptake of advance care planning policies: a modified Delphi study

BACKGROUND

In 2014, the province of Alberta, Canada implemented a province-wide policy and procedures for advance care planning (ACP) and goals of care designation (GCD) across its complex, integrated public healthcare system. This study was conducted to identify and operationalise performance indicators for ACP/GCD to monitor policy implementation success and sustainment of ACP/GCD practice change.

METHODS

A systematic review and environmental scan was conducted to identify potential indicators of ACP/GCD uptake (n=132). A purposive sample of ACP/GCD stakeholders was invited to participate in a modified Delphi study to evaluate, reduce and refine these indicators through a combination of face-to-face meetings and online surveys.

RESULTS

An evidence-informed Donabedian by Institute of Medicine (IOM) framework was adopted as an organising matrix for the indicators in an initial face-to-face meeting. Three online survey rounds reduced and refined the 132 indicators to 18. A final face-to-face meeting operationalised the indicators into a measurable format. Nine indicators, covering 11 of the 18 Donabedian×IOM domains, were operationalised.

CONCLUSIONS

Nine ACP/GCD evidence-informed indicators mapping to 11 of 18 Donabedian×IOM domains were endorsed, and have been operationalised into an online ACP/GCD dashboard. The indicators provide a characterisation of ACP/GCD uptake that could be generalised to other healthcare settings, measuring aspects related to ACP/GCD documentation, patient satisfaction and agreement between medical orders and care received. The final nine indicators reflect the stakeholders' expressed intent to strike a balance between comprehensiveness and feasibility within a large provincial healthcare system.

An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada

BACKGROUND

Quality indicators can be used to identify gaps in care and drive frontline improvement activities. These efforts are important to prevent adverse events in the increasing number of ambulatory patients with advanced kidney disease in Canada, but it is unclear what indicators exist and the components of health care quality they measure.

OBJECTIVE

We sought to identify, categorize, and evaluate quality indicators currently in use across Canada for ambulatory patients with advanced kidney disease.

DESIGN

Environmental scan of quality indicators currently being collected by various organizations.

SETTING

We assembled a 16-member group from across Canada with expertise in nephrology and quality improvement.

PATIENTS

Our scan included indicators relevant to patients with chronic kidney disease in ambulatory care clinics.

MEASUREMENTS

We categorized the identified quality indicators using the Institute of Medicine and Donabedian frameworks.

METHODS

A 4-member panel used a modified Delphi process to evaluate the indicators found during the environmental scan using the American College of Physicians/Agency for Healthcare Research and Quality criteria. The ratings were then shared with the full panel for further comments and approval.

RESULTS

The environmental scan found 28 quality indicators across 7 provinces, with 8 (29%) rated as "necessary" to distinguish high-quality from poor-quality care. Of these 8 indicators, 3 were measured by more than 1 province (% of patients on a statin, number of patients receiving a preemptive transplant, and estimated glomerular filtration rate at dialysis start); no indicator was used by more than 2 provinces. None of the indicators rated as necessary measured timely or equitable care, nor did we identify any measures that assessed the setting in which care occurs (ie, structure measures).

LIMITATIONS

Our list cannot be considered as an exhaustive list of available quality indicators at hand in Canada. Our work focused on quality indicators for nephrology providers and programs, and not indicators that can be applied across primary and specialty providers. We also focused on indicator constructs and not the detailed definitions or their application. Last, our panel does not represent the views of other important stakeholders.

CONCLUSIONS

Our environmental scan provides a snapshot of the scope of quality indicators for ambulatory patients with advanced kidney disease in Canada. This catalog should inform indicator selection and the development of new indicators based on the identified gaps, as well as motivate increased pan-Canadian collaboration on quality measurement and improvement.

TRIAL REGISTRATION

Not applicable as this article is not a systematic review, nor does it report results of a health intervention on human participants.

Selecting home care quality indicators based on the Resident Assessment Instrument-Home Care (RAI-HC) for Switzerland: A public health and healthcare providers' perspective

Background

Despite an increasing importance of home care, quality assurance in this healthcare sector in Switzerland is hardly established. In 2010, Swiss home care quality indicators (QIs) based on the Resident Assessment Instrument-Home Care (RAI-HC) were developed. However, these QIs have not been revised since, although internationally new RAI-HC QIs have emerged. The objective of this study was to assess the appropriateness of RAI-HC QIs to measure quality of home care in Switzerland from a public health and healthcare providers’ perspective.

Methods

First, the appropriateness of RAI-HC QIs, identified in a recent systematic review, was assessed by a multidisciplinary expert panel based on the RAND/UCLA Appropriateness Method taking into account indicators’ public health relevance, potential of influence, and comprehensibility. Second, the QIs selected by the experts were afterwards rated regarding their relevance, potential of influence, and practicability from a healthcare providers’ perspective in focus groups with home care nurses based on the Nominal-Group-Technique. Data were analyzed using median scores and the Disagreement Index.

Results

18 of 43 RAI-HC QIs were rated appropriate by the experts from a public health perspective. The 18 QIs cover clinical, psychosocial, functional and service use aspects. Seven of the 18 QIs were subsequently rated appropriate by home care nurses from a healthcare providers’ perspective. The focus of these QIs is narrow, because three of seven QIs are pain-related. From both perspectives, the majority of RAI-HC QIs were rated inappropriate because of insufficient potential of influence, with healthcare providers rating them more critically.

Conclusions

The study shows that the appropriateness of RAI-HC QIs differs according to the stakeholder perspective and the intended use of QIs. The findings of this study can guide policy-makers and home care organizations on selecting QIs and to critically reflect on their appropriate use.

An Environmental Scan and Evaluation of Home Dialysis Quality Indicators Currently Used in Canada

Background

Quality indicators are important tools to measure and ultimately improve the quality of care provided. Performance measurement may be particularly helpful to grow disciplines that are underutilized and cost-effective, such as home dialysis (peritoneal dialysis and home hemodialysis).

Objective

To identify and catalog home dialysis quality indicators currently used in Canada, as well as to evaluate these indicators as a starting point for future collaboration and standardization of quality indicators across Canada.

Design

An environmental scan of quality indicators from provincial organizations, quality organizations, and stakeholders.

Setting

Sixteen-member pan-Canadian panel with expertise in both nephrology and quality improvement.

Patients

Our environmental scan included indicators relevant to patients on home dialysis.

Measurements

We classified existing indicators based on the Institute of Medicine (IOM) and Donabedian frameworks.

Methods

To evaluate the indicators, a 6-person subcommittee conducted a modified version of the Delphi consensus technique based on the American College of Physicians/Agency for Healthcare Research and Quality criteria. We shared these consensus ratings with the entire 16-member panel for further examination. We rated items from 1 to 9 on 6 domains (1-3 does not meet criteria to 7-9 meets criteria) as well as a global final rating (1-3 unnecessary to 7-9 necessary) to distinguish high-quality from low-quality indicators.

Results

Overall, we identified 40 quality indicators across 7 provinces, with 22 (55%) rated as "necessary" to distinguish high quality from poor quality care. Ten indicators were measured by more than 1 province, and 5 of these indicators were rated as necessary (home dialysis prevalence, home dialysis incidence, anemia target achievement, rates of peritonitis associated with peritoneal dialysis, and home dialysis attrition). None of these indicators captured the IOM domains of timely, patient-centered, or equitable care.

Limitations

The environmental scan is a nonexhaustive list of quality indicators in Canada. The panel also lacked representation from patients, administrators, and allied health professionals.

Conclusions

These results provide Canadian home dialysis programs with a starting point on how to measure quality of care along with the current gaps. This work is an initial and necessary step toward future collaboration and standardization of quality indicators across Canada, so that home dialysis programs can access a smaller number of highly rated balanced indicators to motivate and support patient-centered quality improvement initiatives.

Quality indicators for the diagnosis and management of pediatric tonsillitis

INTRODUCTION

Pediatric tonsillitis is encountered frequently across specialties, and while high quality guidelines exist, there is persistent evidence of care which is not evidence based, including antibiotic overprescribing and surgical practice variability. Quality indicators (QIs) can be utilized for initiatives to improve the quality of care and subsequent patient outcomes. We sought to develop pediatric tonsillitis QIs that are applicable across specialties and that cover aspects of both diagnosis and the spectrum of management options.

METHODS

A guideline-based approach to QI development was employed. Candidate indicators (CIs) were extracted from international guidelines deemed high quality by two reviewers and evaluated by an eleven-member expert panel consisting of otolaryngology - head & neck surgeons, a pediatrician and a family physician. The final QIs were selected utilizing a modified RAND/UCLA appropriateness methodology.

RESULTS

Twenty-six CIs were identified after initial literature review. After the first round of evaluations, the panel agreed on thirteen candidate indicators as appropriate QIs. A subsequent expert panel meeting provided a platform to discuss areas of disagreement, discuss any recently published research, and to brainstorm additional CIs not identified from the guideline extraction. Following the second round of evaluations, the expert panel agreed upon sixteen QIs as appropriate measures of high-quality care.

CONCLUSIONS

This study proposes sixteen QIs developed through a multidisciplinary lens to guide practitioners in the diagnosis and management of pediatric tonsillitis. These QIs can be used to improve transparency, accountability, and provide objective data to assist future quality improvement initiatives.

Practices in Triage and Transfer of Critically Ill Patients: A Qualitative Systematic Review of Selection Criteria

OBJECTIVES

To identify and appraise articles describing criteria used to prioritize or withhold a critical care admission.

DATA SOURCES

PubMed, Embase, Medline, EBM Reviews, and CINAHL Complete databases. Gray literature searches and a manual review of references were also performed. Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines were followed.

STUDY SELECTION

We sought all articles and abstracts of original research as well as local, provincial, or national policies on the topic of ICU resource allocation. We excluded studies whose population of interest was neonatal, pediatric, trauma, or noncritically ill. Screening of 6,633 citations was conducted.

DATA EXTRACTION

Triage and/or transport criteria were extracted, based on type of article, methodology, publication year, and country. An appraisal scale was developed to assess the quality of identified articles. We also developed a robustness score to further appraise the robustness of the evidence supporting each criterion. Finally, all criteria were extracted, evaluated, and grouped by theme.

DATA SYNTHESIS

One-hundred twenty-nine articles were included. These were mainly original research (34%), guidelines (26%), and reviews (21%). Among them, we identified 200 unique triage and transport criteria. Most articles highlighted an exclusion (71%) rather than a prioritization mechanism (17%). Very few articles pertained to transport of critically ill patients (4%). Criteria were classified in one of four emerging themes: patient, condition, physician, and context. The majority of criteria used were nonspecific. No study prospectively evaluated the implementation of its cited criteria.

CONCLUSIONS

This systematic review identified 200 criteria classified within four themes that may be included when devising triage programs including the coronavirus disease 2019 pandemic. We identified significant knowledge gaps where research would assist in improving existing triage criteria and guidelines, aiming to decrease arbitrary decisions and variability.

Clinimetric properties and suitability of selected quality indicators for assessing antibiotic use in hospitalized adults: a multicentre point prevalence study in 24 hospitals in Germany

OBJECTIVES

The capability to measure and monitor the quality of antibiotic prescribing is an important component of antibiotic stewardship (ABS) programmes. Several catalogues of consensus-based structure and process-of-care quality indicators (QIs) have been proposed, but only a few studies have tested and validated ABS QIs in practice tests. This multicentre study determined the clinimetric properties and suitability of a set of 33 process QIs for ABS that had earlier been developed and in part recommended in a German-Austrian hospital ABS practice guideline.

METHODS

Two point prevalence surveys were conducted in a convenience sample of 24 acute care hospitals throughout Germany, and data of all screened adult inpatients with prescription of a systemic antibiotic at a given day (n=4310) were included in the study. For each QI, the following clinimetric properties were assessed: applicability, feasibility, performance, case mix stability and interobserver reliability.

RESULTS

Eighteen QIs were considered sufficiently feasible, applicable and reliable, and had adequate room for improvement. The finally selected QIs primarily cover antibiotic therapy of common infections (bloodstream infection, pneumonia and urinary tract infection), while two of the QIs each address surgical prophylaxis and general aspects of antibiotic administration.

CONCLUSIONS

Practice tests may be important to test the suitability of consensus process-of-care QIs in the field of hospital ABS. The 18 selected QIs considered suitable enough for hospital ABS in this study should be regarded as priority QIs useful for internal quality control and assurance. More research and additional practice tests may be needed to confirm their suitability for external quality assessment schemes.

How well does your massive transfusion protocol perform? A scoping review of quality indicators

BACKGROUND

Management of patients with major haemorrhage often requires urgent administration of multiple blood products, commonly termed a massive transfusion (MT). Clinical practice in these scenarios is supported in part by evidence-based MT guidelines, which typically recommend use of an MT protocol (MTP). MTPs aim to provide practical and specific interpretation of MT guidelines for local institutional use, outlining tasks and pre-configuration of blood product packs to be transfused to provide efficient and evidence-based transfusion management. Institutions can support this aim by the measurement of MTP performance and patient outcomes through collection of quality indicators (QI). Many international guidelines now recommend the routine collection of a range of QIs relating to MT/MTP; however, there is significant variation in procedures and no benchmarks or minimal evidence to guide practice.

MATERIALS AND METHODS

We conducted a scoping review to document and evaluate reported QIs for MTP. We conducted a search of CENTRAL, MEDLINE and EMBASE for published studies from inception until May 14, 2020, that reported at least one MTP QI and use of an MTP or equivalent protocol. Included studies were evaluated using a QI classification system based on current MT QI guidelines and the Donabedian QI framework.

RESULTS

We identified 107 eligible studies. Trauma patients were the most commonly evaluated group, and total blood products transfused and in-hospital mortality were the most commonly reported QIs. Reflecting the lack of international consensus and benchmarks, we found significant variability in the reporting of QIs, which often did not reflect guideline recommendations.

DISCUSSION

Our review highlights the importance of establishing international consensus on prioritised QIs with quantifiable targets that are important to the process of MT.

Identifying Quality Gaps in Preventive Care for Outpatients With Cirrhosis Within a Large, Academic Health Care System

We sought to identify specific gaps in preventive care provided to outpatients with cirrhosis and to determine factors associated with high quality of care (QOC), to guide quality improvement efforts. Outpatients with cirrhosis who received care at a large, academic tertiary health care system in the United States were included. Twelve quality indicators (QIs), including preventive care processes for ascites, esophageal varices, hepatic encephalopathy, hepatocellular carcinoma (HCC), and general cirrhosis care, were measured. QI pass rates were calculated as the proportion of patients eligible for a QI who received that QI during the study period. We performed logistic regression to determine predictors of high QOC (≥ 75% of eligible QIs) and receipt of HCC surveillance. Of the 439 patients, the median age was 63 years, 59% were male, and 19% were Hispanic. The median Model for End-Stage Liver Disease-Sodium score was 11, 64% were compensated, and 32% had hepatitis C virus. QI pass rates varied by individual QIs, but were overall low. For example, 24% received appropriate HCC surveillance, 32% received an index endoscopy for varices screening, and 21% received secondary prophylaxis for spontaneous bacterial peritonitis. In multivariable analyses, Asian race (odds ratio [OR]: 3.7, 95% confidence interval [CI]: 1.3-10.2) was associated with higher QOC, and both Asian race (OR: 3.3, 95% CI: 1.2-9.0) and decompensated status (OR: 2.1, 95% CI: 1.1-4.2) were associated with receipt of HCC surveillance. A greater number of specialty care visits was not associated with higher QOC. Conclusion: Receipt of outpatient preventive cirrhosis QIs was variable and overall low in a diverse cohort of patients with cirrhosis. Variation in care by race/ethnicity and illness trajectory should prompt further inquiry into identifying modifiable factors to standardize care delivery and to improve QOC.

Quality Indicators for the Diagnosis and Management of Acute Bacterial Rhinosinusitis

Background

Acute bacterial rhinosinusitis (ABRS) is a highly prevalent disease that is treated by a variety of specialties, including but not limited to, family physicians, emergency physicians, otolaryngology—head and neck surgeons, infectious disease specialists, and allergy and immunologists. Unfortunately, despite high-quality guidelines, variable and substandard care continues to be demonstrated in the treatment of ABRS.

Objective

This study aimed to develop ABRS-specific quality indicators (QIs) to evaluate the diagnosis and management that reduces symptoms, improves quality of life, and prevents complications.

Methods

A guideline-based approach, proposed by Kötter et al., was used to develop QIs for ABRS. Candidate indicators (CIs) were extracted from 4 guiding documents and evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool. Each CI and its supporting evidence was summarized and reviewed by an expert panel based on validity, reliability, and feasibility of measurement. Final QIs were selected from CIs utilizing the modified RAND/University of California at Los Angeles appropriateness methodology.

Results

Twenty-nine CIs were identified after literature review and evaluated by our panel. Of these, 5 CIs reached consensus as being appropriate QIs, with 1 requiring additional discussion. After a second round of evaluations, the panel selected 7 QIs as appropriate measures of high-quality care.

Conclusion

This study proposes 7 QIs for the diagnosis and management of patients with ABRS. These QIs can serve multiple purposes, including documenting the quality of care; comparing institutions and providers; prioritizing quality improvement initiatives; supporting accountability, regulation, and accreditation; and determining pay for performance initiatives.

Developing and evaluating packages to support implementation of quality indicators in general practice: the ASPIRE research programme, including two cluster RCTs

Background

Dissemination of clinical guidelines is necessary but seldom sufficient by itself to ensure the reliable uptake of evidence-based practice. There are further challenges in implementing multiple clinical guidelines and clinical practice recommendations in the pressurised environment of general practice.

Objectives

We aimed to develop and evaluate an implementation package that could be adapted to support the uptake of a range of clinical guideline recommendations and be sustainably integrated within general practice systems and resources. Over five linked work packages, we developed ‘high-impact’ quality indicators to show where a measurable change in clinical practice can improve patient outcomes (work package 1), analysed adherence to selected indicators (work package 2), developed an adaptable implementation package (work package 3), evaluated the effects and cost-effectiveness of adapted implementation packages targeting four indicators (work package 4) and examined intervention fidelity and mechanisms of action (work package 5).

Setting and participants

Health-care professionals and patients from general practices in West Yorkshire, UK.

Design

We reviewed recommendations from existing National Institute for Health and Care Excellence clinical guidance and used a multistage consensus process, including 11 professionals and patients, to derive a set of ‘high-impact’ evidence-based indicators that could be measured using routinely collected data (work package 1). In 89 general practices that shared data, we found marked variations and scope for improvement in adherence to several indicators (work package 2). Interviews with 60 general practitioners, practice nurses and practice managers explored perceived determinants of adherence to selected indicators and suggested the feasibility of adapting an implementation package to target different indicators (work package 3). We worked with professional and patient panels to develop four adapted implementation packages. These targeted risky prescribing involving non-steroidal anti-inflammatory and antiplatelet drugs, type 2 diabetes control, blood pressure control and anticoagulation for atrial fibrillation. The implementation packages embedded behaviour change techniques within audit and feedback, educational outreach and (for risky prescribing) computerised prompts. We randomised 178 practices to implementation packages targeting either diabetes control or risky prescribing (trial 1), or blood pressure control or anticoagulation (trial 2), or to a further control (non-intervention) group, and undertook economic modelling (work package 4). In trials 1 and 2, practices randomised to the implementation package for one indicator acted as control practices for the other package, and vice versa. A parallel process evaluation included a further eight practices (work package 5).

Main outcome measures

Trial primary end points at 11 months comprised achievement of all recommended levels of glycated haemoglobin, blood pressure and cholesterol; risky prescribing levels; achievement of recommended blood pressure; and anticoagulation prescribing.

Results

We recruited 178 (73%) out of 243 eligible general practices. We randomised 80 practices to trial 1 (40 per arm) and 64 to trial 2 (32 per arm), with 34 non-intervention controls. The risky prescribing implementation package reduced risky prescribing (odds ratio 0.82, 97.5% confidence interval 0.67 to 0.99; p = 0.017) with an incremental cost-effectiveness ratio of £2337 per quality-adjusted life-year. The other three packages had no effect on primary end points. The process evaluation suggested that trial outcomes were influenced by losses in fidelity throughout intervention delivery and enactment, and by the nature of the targeted clinical and patient behaviours.

Limitations

Our programme was conducted in one geographical area; however, practice and patient population characteristics are otherwise likely to be sufficiently diverse and typical to enhance generalisability to the UK. We used an ‘opt-out’ approach to recruit general practices to the randomised trials. Subsequently, our trial practices may have engaged with the implementation package less than if they had actively volunteered. However, this approach increases confidence in the wider applicability of trial findings as it replicates guideline implementation activities under standard conditions.

Conclusions

This pragmatic, rigorous evaluation indicates the value of an implementation package targeting risky prescribing. In broad terms, an adapted ‘one-size-fits-all’ approach did not consistently work, with no improvement for other targeted indicators.

Future work

There are challenges in designing ‘one-size-fits-all’ implementation strategies that are sufficiently robust to bring about change in the face of difficult clinical contexts and fidelity losses. We recommend maximising feasibility and ‘stress testing’ prior to rolling out interventions within a definitive evaluation. Our programme has led on to other work, adapting audit and feedback for other priorities and evaluating different ways of delivering feedback to improve patient care.

Trial registration

Current Controlled Trials ISRCTN91989345.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 8, No. 4. See the NIHR Journals Library website for further project information.

Risk Prediction Accuracy Differs for Transferred and Nontransferred Emergency General Surgery Cases in the ACS-NSQIP

BACKGROUND

Risk prediction accuracy of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) Surgical Risk Calculator has been shown to differ between emergency and elective surgery. Benchmarking methods of clinical performance require accurate risk estimation, and current methods rarely account for admission source; therefore, our goal was to assess whether the ACS-NSQIP predicts mortality comparably between transferred and nontransferred emergency general surgery (EGS) cases.

MATERIALS AND METHODS

This is a retrospective study using the ACS-NSQIP database from 2005 to 2014including all inpatients who underwent one of seven previously described EGS procedures. The admission source was classified as directly admitted versus transferred from an outside emergency room or an acute care facility. We compared the accuracy of ACS-NSQIP-predicted mortality probabilities using the observed-to-expected (O:E) ratio and Brier score. A subgroup analysis was performed to compare accuracy of high-risk and low-risk procedures.

RESULTS

A total of 206,103 EGS admissions were identified, of which 6.97% were transfers. Overall mortality was 3.26% for the entire cohort and 10.24% within the transfer group. The O:E ratios generated by ACS-NSQIP models differed between transferred patients (O:E = 1.0, 95% confidence interval = 0.97-1.02) and nontransferred patients (O:E = 1.12, 95% confidence interval = 1.09-1.14). The Brier score for transferred patients was greater than that for nontransferred patients (0.063 versus 0.018, respectively) showing higher accuracy for nontransferred patients.

CONCLUSIONS

The ACS-NSQIP risk estimates used for benchmarking differ between transferred and nontransferred EGS cases. Analyses of the Brier score by the ACS-NSQIP risk calculator demonstrated inferior prediction for transferred patients. This increased burden on accepting institutions will have an impact on quality metrics and should be considered for benchmarking of clinical performance.

Quality indicators and their regular use in clinical practice: results from a survey among users of two cardiovascular National Registries in Sweden

Objective

To examine the regular use of quality indicators from Swedish cardiovascular National Quality Registries (NQRs) by clinical staff; particularly differences in use between the two NQRs and between nurses and physicians.

Design

Cross-sectional online survey study.

Setting

Two Swedish cardiovascular NQRs: (a) Swedish Heart Failure Registry and (b) Swedeheart.

Participants

Clinicians (n =185; 70% nurses, 26% physicians) via the NQRs' email networks.

Main Outcome Measures

Frequency of NQR use for (a) producing healthcare activity statistics; (b) comparing results between similar departments; (c) sharing results with colleagues; (d) identifying areas for quality improvement (QI); (e) surveilling the impact of QI efforts; (f) monitoring effects of implementation of new treatment methods; (g) doing research and (h) educating and informing healthcare professionals and patients.

Results

Median use of NQRs was 10 times a year (25th and 75th percentiles range: 3-23 times/year). Quality indicators from the NQRs were used mainly for producing healthcare activity statistics. Median use of Swedeheart was six times greater than Swedish Heart Failure Registry (SwedeHF; P < 0.000). Physicians used the NQRs more than twice as often as nurses (18 vs. 7.5 times/year; P < 0.000) and perceived NQR work more often as meaningful. Around twice as many Swedeheart users had the role to participate in data analysis and in QI efforts compared to SwedeHF users.

Conclusions

Most respondents used quality indicators from the two cardiovascular NQRs infrequently (<3 times/year). The results indicate that linking registration of quality indicators to using them for QI activities increases their routine use and makes them meaningful tools for professionals.