Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation

Quality indicators for integrating oncology and home palliative care in Japan: modified Delphi study

CONTEXT

Home palliative care service increases the chance of dying at home, particularly for patients with advanced cancer, but late referrals to home palliative care services still exist. Indicators for evaluating programs that can facilitate the integration of oncology and home palliative care have not been defined.

OBJECTIVES

This study developed quality indicators for the integration of oncology and home palliative care in Japan.

METHODS

We conducted a systematic literature review (Databases included CENTRAL, MEDLINE, EMBASE, and Emcare) and a modified Delphi study to develop the quality indicators. Panelists rated a potential list of indicators using a 9-point scale over three rounds according to two criteria: appropriateness and feasibility. The criterion for the adoption of candidate indicators was set at a total mean score of 7 or more. Final quality indicators with no disagreement were included.

RESULTS

Of the 973 publications in our initial search, 12 studies were included. The preliminary list of quality indicators by systematic literature review comprised 50 items. In total, 37 panelists participated in the modified Delphi study. Ultimately, 18 indicators were identified from the following domains: structure in cancer hospitals, structure in home palliative care services, the process of home palliative care service delivery, less aggressive end-of-life care, patient's psychological comfort, caregiver's psychological comfort, and patient's satisfaction with home palliative care service.

CONCLUSION

Comprehensive quality indicators for the integration of oncology and home palliative care were identified. These indicators may facilitate interdisciplinary collaboration between professional healthcare providers in both cancer hospitals and home palliative care services.

Consensus quality indicators for monitoring multiple sclerosis

Multiple sclerosis (MS) as a chronic, degenerative autoimmune disease of the central nervous system has a longitudinal and heterogeneous course with increasing treatment options and risk profiles requiring constant monitoring of a growing number of parameters. Despite treatment guidelines, there is a lack of strategic and individualised monitoring pathways, including respective quality indicators (QIs). To address this, we systematically developed transparent, traceable, and measurable QIs for MS monitoring. Through literature review, expert discussions, and consensus-building, existing QIs were identified and refined. In a two-stage online Delphi process involving MS specialists (on average 53 years old and with 25 years of professional experience), the QIs were evaluated for content, clarity, and intelligibility, resulting in a set of 24 QIs and checklists to assess the quality of care. The final QIs provide a structured approach to document, monitor, and enhance the quality of care for people with MS across their treatment journey.

Evidence-based interventions for identifying candidate quality indicators to assess quality of care in diabetic foot clinics: a scoping review

BACKGROUND

Foot ulcers in people with diabetes are a serious complication requiring a complex management and have a high societal impact. Quality monitoring systems to optimize diabetic foot care exist, but a formal and more evidence-based approach to develop quality indicators (QIs) is lacking. We aimed to identify a set of candidate indicators for diabetic foot care by adopting an evidence-based methodology.

METHODS

A systematic search was conducted across four academic databases: PubMed, Embase CINAHL and Cochrane Library. Studies that reported evidence-based interventions related to organization or delivery of diabetic foot care were searched. Data from the eligible studies were summarized and used to formulate process and structure indicators. The evidence for each candidate QI was described in a methodical and transparent manner. The review process was reported according to the "Preferred Reported Items for Systematic reviews and Meta-Analysis" (PRISMA) statements and its extension for scoping reviews.

RESULTS

In total, 981 full-text articles were screened, and 322 clinical studies were used to formulate 42 candidate QIs.

CONCLUSIONS

An evidence-based approach could be used to select candidate indicators for diabetic foot ulcer care, relating to the following domains: wound healing interventions, peripheral artery disease, offloading, secondary prevention, and interventions related to organization of care. In a further step, the feasibility of the identified set of indicators will be assessed by a multidisciplinary panel of diabetic foot care stakeholders.

Developing an initial set of quality indicators for chiropractic care: a scoping review

BACKGROUND

Quality indicators are standardized, evidence-based measures of health care quality. Currently, there is no basic set of quality indicators for chiropractic care published in peer-reviewed literature. The goal of this research is to develop a preliminary set of quality indicators, measurable with administrative data.

METHODS

We conducted a scoping review searching PubMed/MEDLINE, CINAHL, and Index to Chiropractic Literature databases. Eligible articles were published after 2011, in English, developing/reporting best practices and clinical guidelines specifically developed for, or directly applicable to, chiropractic care. Eligible non-peer-reviewed sources such as quality measures published by the Centers for Medicare and Medicaid Services and the Royal College of Chiropractors quality standards were also included. Following a stepwise eligibility determination process, data abstraction identified specific statements from included sources that can conceivably be measured with administrative data. Once identified, statements were transformed into potential indicators by: 1) Generating a brief title and description; 2) Documenting a source; 3) Developing a metric; and 4) Assigning a Donabedian category (structure, process, outcome). Draft indicators then traversed a 5-step assessment: 1) Describes a narrowly defined structure, process, or outcome; 2) Quantitative data can conceivably be available; 3) Performance is achievable; 4) Metric is relevant; 5) Data are obtainable within reasonable time limits. Indicators meeting all criteria were included in the final set.

RESULTS

Literature searching revealed 2562 articles. After removing duplicates and conducting eligibility determination, 18 remained. Most were clinical guidelines (n = 10) and best practice recommendations (n = 6), with 1 consensus and 1 clinical standards development study. Data abstraction and transformation produced 204 draft quality indicators. Of those, 57 did not meet 1 or more assessment criteria. After removing duplicates, 70 distinct indicators remained. Most indicators matched the Donabedian category of process (n = 35), with 31 structure and 4 outcome indicators. No sources were identified to support indicator development from patient perspectives.

CONCLUSIONS

This article proposes a preliminary set of 70 quality indicators for chiropractic care, theoretically measurable with administrative data and largely obtained from electronic health records. Future research should assess feasibility, achieve stakeholder consensus, develop additional indicators including those considering patient perspectives, and study relationships with clinical outcomes.

TRIAL REGISTRATION

Open Science Framework, https://osf.io/t7kgm.

Nationwide consensus on quality indicators to assess glaucoma care: A modified Delphi approach

PURPOSE

Performance assessments are essential to tracking and improving quality in health care systems. Key aspects of the care process that act as indicators must be measured in order to gain an in-depth understanding of a care unit's operation. Without standardized quality indicators (QIs), it is difficult to characterize and compare the abilities of institutions to achieve excellence. The aim of this study is to reach a consensus among glaucoma specialists concerning the development of a set of QIs to assess the performance of glaucoma care units.

METHODS

A two-round Delphi technique was performed among glaucoma specialists in Portugal, using a 7-point Likert scale. Fifty-three initial statements (comprising process, structure, and outcome indicators) were evaluated and participants had to agree on which ones would be part of the final set of QIs.

RESULTS

By the end of both rounds, 28 glaucoma specialists reached consensus on 30/53 (57%) statements, including 19 (63%) process indicators (mainly relating to the proper implementation of complementary exams and the setting of follow-up intervals), 6 (20%) structure indicators, and 5 (17%) outcome indicators. Of the indicators that were part of the final list, functional and structural aspects of glaucoma progression and the availability of surgical/laser procedures were the most prevalent.

CONCLUSIONS

A set of 30 QIs for measuring the performance of glaucoma units was developed using a consensus methodology involving experts in the field. Their use as measurement standards would provide important information about unit operations and allow further implementation of quality improvements.

Interventions to Promote End-of-Life Conversations: A Systematic Review and Meta-Analysis

CONTEXT

Although several interventions aimed to promote end-of-life conversations are available, it is unclear whether and how these affect delivery of end-of-life conversations. Measuring the processes associated with high-quality end-of-life care may trigger improvement.

OBJECTIVES

To estimate the effect of interventions aimed to promote end-of-life conversations in clinical encounters with patients with advanced chronic or terminal illness or their family, on process indicators of end-of-life conversations.

METHODS

Systematic review with meta-analysis (PROSPERO no. CRD42021289471). Four databases (PubMed, CINAHL, PsycINFO, and Scopus) were searched up to September 30, 2021. The primary outcomes were any process indicators of end-of-life conversations. Results of pairwise meta-analyses were presented as Risk Ratio (RR) for occurrence, standardized mean difference (SMD) for quality and ratio of means (ROM) for duration. Meta-analysis was not performed when fewer than four studies were available.

RESULTS

A total of 4,663 articles were scanned. Eighteen studies were included in the systematic review and 16 entered at least one meta-analysis: documented occurrence (n = 8), patient-reported occurrence (n = 4), patient-reported-quality (n = 4), duration (n = 4). There was significant variability in settings, patients' clinical conditions, and professionals. No significant effect of interventions on documented occurrence (RR 1.54, 95% CI 0.84-2.84; I2 91%), patient-reported occurrence (RR 1.52, 95% CI 0.80-2.91; I2 95%), patient-reported quality (SMD 0.83, 95% CI -1.06 to 2.71; I2 99%), or duration (ROM 1.20, 95% CI 0.95-1.51; I2 65%) of end-of-life conversations was found. Data on frequency were conflicting. Interventions targeting multiple stakeholders promoted earlier and more comprehensive conversations.

CONCLUSION

Heterogeneity was considerable, but findings suggest no significant effect of interventions on occurrence, patient-reported quality and duration of end-of-life conversations. Nevertheless, we found indications for interventions targeting multiple stakeholders to promote earlier and more comprehensive conversations.

Development of Quality Indicators for Geriatric Home Enteral Nutrition (HEN) Services

The evidence on the safety, efficacy and patient centeredness of Home Enteral Nutrition (HEN) services is scarce. In 2015, we carried out a search of the literature to identify specific indicators for HEN services as tools to be used to assess the quality of INRCA HEN services. No specific indicators for HEN services were found. Through a subsequent search of the literature, we have identified the appropriate methodology to define quality indicators and developed eight (8) specific indicators to track the quality of our HEN service for geriatric patients. Following Donabiedan's classification, we have defined two structure indicators, two process indicators and four outcome indicators that are presented in this manuscript. Though they may be used to make a comparison of HEN services for geriatric patients and to monitor the quality of therapy provided at patients' homes, the definition of quality system indicators for HEN services requires the additional joint efforts of experts in the field of nutrition and the scientific community for their validation.

Identifying Quality Indicators for the Care of Hospitalized Injured Older Adults: A Scoping Review of the Literature

OBJECTIVES

Older adults represent more than 50% of trauma admissions in many high-income countries. Furthermore, they are at increased risk for complications, resulting in worse health outcomes than younger adults and a significant health care utilization burden. Quality indicators (QIs) are used to assess the quality of care in trauma systems, but few QIs reflect responses to older patients' specific needs. We aimed to (1) identify QIs used to assess acute hospital care for injured older patients, (2) assess support for identified QIs and, (3) identify gaps in existing QIs.

DESIGN

Scoping review of the scientific and gray literature.

METHODS

Selection and data extraction were performed by 2 independent reviewers. The level of support was assessed by the number of sources reporting QIs and whether they were developed according to scientific evidence, expert consensus, and patients' perspectives.

RESULTS

Of 10,855 identified studies, 167 were eligible. Among 257 different QIs identified, 52% were hip fracture specific. Gaps were identified for head injuries, rib, and pelvic ring fractures. Although 61% of QIs assessed care processes, 21% and 18% focused on structures and outcomes, respectively. Although most QIs were based on literature reviews and/or expert consensus, patients' perspective was rarely accounted for. The 15 QIs with the highest level of support included minimum time between emergency department arrival and ward admission, minimum time to surgery for fractures, assessment by a geriatrician, orthogeriatric review for hip fracture patients, delirium screening, prompt and appropriate analgesia, early mobilizations, and physiotherapy.

CONCLUSION AND IMPLICATIONS

Multiple QIs were identified, but their level of support was limited, and important gaps were identified. Future work should focus on achieving consensus for a set of QIs to assess the quality of trauma care to older adults. Such QIs could be used for quality improvement and ultimately improve outcomes for injured older adults.

Methods for developing quality indicators for evidence implementation: a scoping review protocol

OBJECTIVE

The aim of this scoping review is to locate, identify, and understand the extent and type of evidence in relation to the development of quality indicators within evidence implementation health care programs.

INTRODUCTION

Health care organizations evaluate care using quality improvement initiatives, which are based on quality indicators that are clearly defined and measure what they are intended to, based on structures, processes, or outcomes of care. However, the development of quality indicators is an area plagued by inconsistency and issues in terms of their pragmatic use in the health care context. Inconsistency when using terminology related to quality improvement has highlighted issues with distinct definitions, and many terms appear to be used interchangeably. This leads to confusion and a lack of clarity in what these terms are actually describing with regard to measures of quality in health care.

INCLUSION CRITERIA

The proposed review will consider studies, guidelines, manuals, evidence syntheses, and other relevant literature that examine the key concepts, terms, or definitions used in the development of quality indicators and that identify the methods or frameworks used in the development approaches across the international health care setting.

METHODS

The search strategy will aim to locate both published and unpublished documents, using a 3-step search strategy. Results of the search, study inclusion, data extraction, and analysis and presentation of results will be conducted by 2 independent reviewers, in accordance with JBI's methodology for conducting scoping reviews.Findings will be presented in tables or visual charts, accompanied by a narrative summary.

DETAILS OF THIS REVIEW PROJECT ARE AVAILABLE AT

Open Science Framework https://osf.io/54q8j.

DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey

OBJECTIVE

The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact.

METHODS

We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance.

RESULTS

We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines.

CONCLUSIONS

Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to "de-implement" traditional measures that do not uphold DORA principles yet are still in use.

Consensus on how to optimise patient/family engagement in hospital planning and improvement: a Delphi survey

OBJECTIVE

Patient and family engagement (PE) in health service planning and improvement is widely advocated, yet little prior research offered guidance on how to optimise PE, particularly in hospitals. This study aimed to engage stakeholders in generating evidence-informed consensus on recommendations to optimise PE.

DESIGN

We transformed PE processes and resources from prior research into recommendations that populated an online Delphi survey.

SETTING AND PARTICIPANTS

Panellists included 58 persons with PE experience including: 22 patient/family advisors and 36 others (PE managers, clinicians, executives and researchers) in round 1 (100%) and 55 in round 2 (95%).

OUTCOME MEASURES

Ratings of importance on a seven-point Likert scale of 48 strategies organised in domains: engagement approaches, strategies to integrate diverse perspectives, facilitators, strategies to champion engagement and hospital capacity for engagement.

RESULTS

Of 50 recommendations, 80% or more of panellists prioritised 32 recommendations (27 in round 1, 5 in round 2) across 5 domains: 5 engagement approaches, 4 strategies to identify and integrate diverse patient/family advisor perspectives, 9 strategies to enable meaningful engagement, 9 strategies by which hospitals can champion PE and 5 elements of hospital capacity considered essential for supporting PE. There was high congruence in rating between patient/family advisors and healthcare professionals for all but six recommendations that were highly rated by patient/family advisors but not by others: capturing diverse perspectives, including a critical volume of advisors on committees/teams, prospectively monitoring PE, advocating for government funding of PE, including PE in healthcare worker job descriptions and sharing PE strategies across hospitals.

CONCLUSIONS

Decision-makers (eg, health system policy-makers, hospitals executives and managers) can use these recommendations as a framework by which to plan and operationalise PE, or evaluate and improve PE in their own settings. Ongoing research is needed to monitor the uptake and impact of these recommendations on PE policy and practice.

Towards a harmonised framework for developing quality of care indicators for global health: a scoping review of existing conceptual and methodological practices

OBJECTIVES

Despite significant advances in the science of quality of care measurement over the last decade, approaches to developing quality of care indicators for global health priorities are not clearly defined. We conducted a scoping review of concepts and methods used to develop quality of healthcare indicators to better inform ongoing efforts towards a more harmonised approach to quality of care indicator development in global health.

METHODS

We conducted a systematic search of electronic databases, grey literature and references for articles on developing quality of care indicators for routine monitoring in all healthcare settings and populations, published in English between 2010 and 2020. We used well-established methods for article screening and selection, data extraction and management. Results were summarised using a descriptive analysis and a narrative synthesis.

RESULTS

The 221 selected articles were largely from high-income settings (89%), particularly the USA (46%), Canada (9%), UK (9%) and Europe (17%). Quality of care indicators were developed mainly for healthcare providers (56%), for benchmarking or quality assurance (37%) and quality improvement (29%), in hospitals (32%) and primary care (26%), across many diseases. The terms 'quality indicator' and 'quality measure' were the most frequently encountered terms (50% and 21%, respectively). Systematic approaches for quality of care indicator development emerged within national quality of care systems or through cross-country collaborations in high-income settings. Maternal, neonatal and child health (33%), mental health (26%) and primary care (57%) studies applied most components of systematic approaches, but not consistently or rigorously.

DISCUSSION

The current evidence shows variations in concepts and approaches to developing quality of care indicators, with development and application mainly in high-income countries.

CONCLUSION

Additional efforts are needed to propose 'best-practice' conceptual frameworks and methods for developing quality of care indicators to improve their utility in global health measurement.

Quality of Care Indicators for Hospital Physical Therapy Units: A Systematic Review

OBJECTIVE

The purpose of this review was to identify quality indicators described in the literature that may be used as quality measures in hospital physical therapy units.

METHODS

The following sources were searched for quality indicators or articles: Web of Science, MEDLINE, IBECS, Latin American and Caribbean Health Sciences Literature, Cumulative Index of Nursing and Allied Health, Academic Search Complete, SportDiscus, SciELO, PsychINFO, Consejo Superior de Investigaciones Cientificas, and Scopus databases; the Agency for Healthcare Research and Quality, National Health System Indicator Portal, Joint Commission on Accreditation of Healthcare Organizations, and Organisation for Economic Co-operation and Development websites; and the National Quality Forum's measures inventory tool. Search terms included "quality indicator," "quality measure," "physiotherapy," and "physical therapy." Inclusion criteria were articles written in English, Spanish, French, or Portuguese aimed at measuring the quality of care in hospital physical therapy units. Evidence-based indicators with an explicit formula were extracted by 2 independent reviewers and then classified using the structure-process-outcome model, quality domain, and categories defined by a consensus method.

RESULTS

Of the 176 articles identified, only 19 met the criteria. From these articles and from the indicator repository searches, 178 clinical care indicators were included in the qualitative synthesis and presented in this paper. Process and outcome measures were prevalent, and 5 out of the 6 quality domains were represented. No efficiency measures were identified. Moreover, structure indicators, equity and accessibility indicators, and indicators in the cardiovascular and circulatory, mental health, pediatrics, and intensive care categories were underrepresented.

CONCLUSIONS

A broad selection of quality indicators was identified from international resources, which can be used to measure the quality of physical therapy care in hospital units.

IMPACT

This review identified 178 quality of care indicators that can be used in clinical practice monitoring and quality improvement of hospital physical therapy units. The results highlight a lack of accessibility, equity, and efficiency measures for physical therapy units.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

Using data linkage for national surveillance of clinical quality indicators for dementia care among Australian aged care users

Care quality has important implications for people with dementia. We examined trends and geographical variation of four clinical quality indicators (CQIs) in Australia. This retrospective cohort study included all people with dementia using Australian government-subsidised aged care in 2008-2016 (n = 373,695). Quality indicator data were derived from linked national aged care, health, and pharmaceutical datasets. Negative binomial regression modelling assessed trends in CQI performance over time (2011-2016) and funnel plots examined geographical variation in performance. The incidence rate of antipsychotic medicine dispensing decreased slightly from 1.17/1000 person-days to 1.07/1000 person-days (adjusted incidence rate ratio (aIRR) = 0.98, 95%CI 0.98-0.99). Cholinesterase inhibitors and memantine dispensing did not change (aIRR = 1.02, 95%CI 1.00-1.04), while exposure to high sedative load increased slightly from 1.39/1000 person-days to 1.44/1000 person-days (aIRR = 1.01, 95%CI 1.00-1.01). Dementia and delirium-related hospitalisations increased slightly from 0.17/1000 person-days to 0.18/1000 person-days (aIRR = 1.02, 95%CI 1.01-1.03). There was marked variation in cholinesterase inhibitor and memantine dispensing by geographical area (0-41%). There has been little change in four indicators of dementia care quality in Australian aged care users over time. Cholinesterase inhibitor and memantine dispensing varied substantially by geographical region. Existing strategies to improve national performance on these indicators appear to be insufficient, despite the significant impact of these indicators on outcomes for people with dementia.

Development of a New Tool for Better Social Recognition of Cancer Information and Support Activities Under the National Cancer Control Policy in Japan

Under the recent trend of an increasing number of cancer survivors, there is a need to devise measures for visualization of medical care and public health programs related to cancer control, which will enable better overview of the activities at hospitals and local communities and allow various stakeholders to share the information about such activities. The aim of this study was to propose a new tool for proper implementation of cancer information and support programs provided under the national cancer control policy in Japan. Considering 5 elements reported by Handler et al (macro context, mission, structural capacity, processes, and outcomes), we conducted the focus group discussions to confirm the goals of activities of Cancer Information and Support Centers. Eventually, 2 final goals ("reduction in the number of patients/families having difficulties related to cancer" and "being able to live at ease even after diagnosis of cancer") were identified, accompanied by 5 semifinal goals and 16 prerequisite conditions needed to achieve the final goals, as well as the necessary states and the activity indicators corresponding to them. This tool was utilized by 180 (42.7%) of 422 cancer care hospitals designated by the government of Japan (designated cancer care hospitals [DCCHs]) in 2016 and by 336 (77.1%) of 436 DCCHs in 2018, which were the data at 6 months and 3 years after introduction of the tool, respectively. Thus, the tool for evaluating the Cancer Information and Support Centers' activity presented here is expected to stimulate the stakeholders involved in providing supports in various fields of each local community, to share the final goals, to evaluate the status of their achievement, and to further advance their own activities.

An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada

BACKGROUND

Quality indicators can be used to identify gaps in care and drive frontline improvement activities. These efforts are important to prevent adverse events in the increasing number of ambulatory patients with advanced kidney disease in Canada, but it is unclear what indicators exist and the components of health care quality they measure.

OBJECTIVE

We sought to identify, categorize, and evaluate quality indicators currently in use across Canada for ambulatory patients with advanced kidney disease.

DESIGN

Environmental scan of quality indicators currently being collected by various organizations.

SETTING

We assembled a 16-member group from across Canada with expertise in nephrology and quality improvement.

PATIENTS

Our scan included indicators relevant to patients with chronic kidney disease in ambulatory care clinics.

MEASUREMENTS

We categorized the identified quality indicators using the Institute of Medicine and Donabedian frameworks.

METHODS

A 4-member panel used a modified Delphi process to evaluate the indicators found during the environmental scan using the American College of Physicians/Agency for Healthcare Research and Quality criteria. The ratings were then shared with the full panel for further comments and approval.

RESULTS

The environmental scan found 28 quality indicators across 7 provinces, with 8 (29%) rated as "necessary" to distinguish high-quality from poor-quality care. Of these 8 indicators, 3 were measured by more than 1 province (% of patients on a statin, number of patients receiving a preemptive transplant, and estimated glomerular filtration rate at dialysis start); no indicator was used by more than 2 provinces. None of the indicators rated as necessary measured timely or equitable care, nor did we identify any measures that assessed the setting in which care occurs (ie, structure measures).

LIMITATIONS

Our list cannot be considered as an exhaustive list of available quality indicators at hand in Canada. Our work focused on quality indicators for nephrology providers and programs, and not indicators that can be applied across primary and specialty providers. We also focused on indicator constructs and not the detailed definitions or their application. Last, our panel does not represent the views of other important stakeholders.

CONCLUSIONS

Our environmental scan provides a snapshot of the scope of quality indicators for ambulatory patients with advanced kidney disease in Canada. This catalog should inform indicator selection and the development of new indicators based on the identified gaps, as well as motivate increased pan-Canadian collaboration on quality measurement and improvement.

TRIAL REGISTRATION

Not applicable as this article is not a systematic review, nor does it report results of a health intervention on human participants.

An Environmental Scan of Canadian Quality Metrics for Patients on In-Center Hemodialysis

Background:

Quality metrics or indicators help guide quality improvement work by reporting on measurable aspects of health care upon which improvement efforts can focus. For recipients of in-center hemodialysis (ICHD) in Canada, it is unclear what ICHD quality indicators exist and whether they adequately cover different domains of health care quality.

Objectives:

To identify and evaluate current Canadian ICHD quality metrics to document a starting point for future collaborations and standardization of quality improvement in Canada.

Design:

Environmental scan of quality metrics in ICHD, and subsequent indicator evaluation using a modified Delphi approach.

Setting:

Canadian ICHD units.

Participants:

Sixteen-member pan-Canadian working group with expertise in ICHD and quality improvement.

Measurements:

We classified the existing indicators based on the Institute of Medicine (IOM) and Donabedian frameworks.

Methods:

Each metric was rated by a 5-person subcommittee using a modified Delphi approach based on the American College of Physicians/Agency for Healthcare Research and Quality criteria. We shared these consensus ratings with the entire 16-member panel for additional comments.

Results:

We identified 27 metrics that are tracked across 8 provinces, with only 9 (33%) tracked by multiple provinces (ie, more than 1 province). We rated 9 metrics (33%) as “necessary” to distinguish high-quality from low-quality care, of which only 2 were tracked by multiple provinces (proportion of patients by primary access and rate of vascular access-related bloodstream infections). Most (16/27, 59%) indicators assessed the IOM domains of safe or effective care, and none of the “necessary” indicators measured the IOM domains of timely, patient-centered, or equitable care.

Limitations:

The environmental scan is a nonexhaustive list of quality indicators in Canada. The panel also lacked representation from patients, administrators, and allied health professionals, with more representation from academic sites.

Conclusions:

Quality indicators in Canada mainly focus on safe and effective care, with little provincial overlap. These results highlight current gaps in quality of care measurement for ICHD, and this initial work should provide programs with a starting point to combine highly rated indicators with newly developed indicators into a concise balanced scorecard that supports quality improvement initiatives across all aspects of ICHD care.

Trial Registration:

not applicable.

Improving the quality of person-centred healthcare from the patient perspective: development of person-centred quality indicators

IMPORTANCE

International efforts are being made towards a person-centred care (PCC) model, but there are currently no standardised mechanisms to measure and monitor PCC at a healthcare system level. The use of metrics to measure PCC can help to drive the changes needed to improve the quality of healthcare that is person centred.

OBJECTIVE

To develop and validate person-centred care quality indicators (PC-QIs) measuring PCC at a healthcare system level through a synthesis of the evidence and a person-centred consensus approach to ensure the PC-QIs reflect what matters most to people in their care.

METHODS

Existing indicators were first identified through a scoping review of the literature and an international environmental scan. Focus group discussions with diverse patients and caregivers and interviews with clinicians and experts in quality improvement allowed us to identify gaps in current measurement of PCC and inform the development of new PC-QIs. A set of identified and newly developed PC-QIs were subsequently refined by Delphi consensus process using a modified RAND/UCLA Appropriateness Method. The international consensus panel consisted of patients, family members, community representatives, clinicians, researchers and healthcare quality experts.

RESULTS

From an initial 39 unique evidence-based PC-QIs identified and developed, the consensus process yielded 26 final PC-QIs. These included 7 related to structure, 16 related to process, 2 related to outcome and 1 overall global PC-QI.

CONCLUSIONS

The final 26 evidence-based and person-informed PC-QIs can be used to measure and evaluate quality incorporating patient perspectives, empowering jurisdictions to monitor healthcare system performance and evaluate policy and practice related to PCC.

How well does your massive transfusion protocol perform? A scoping review of quality indicators

BACKGROUND

Management of patients with major haemorrhage often requires urgent administration of multiple blood products, commonly termed a massive transfusion (MT). Clinical practice in these scenarios is supported in part by evidence-based MT guidelines, which typically recommend use of an MT protocol (MTP). MTPs aim to provide practical and specific interpretation of MT guidelines for local institutional use, outlining tasks and pre-configuration of blood product packs to be transfused to provide efficient and evidence-based transfusion management. Institutions can support this aim by the measurement of MTP performance and patient outcomes through collection of quality indicators (QI). Many international guidelines now recommend the routine collection of a range of QIs relating to MT/MTP; however, there is significant variation in procedures and no benchmarks or minimal evidence to guide practice.

MATERIALS AND METHODS

We conducted a scoping review to document and evaluate reported QIs for MTP. We conducted a search of CENTRAL, MEDLINE and EMBASE for published studies from inception until May 14, 2020, that reported at least one MTP QI and use of an MTP or equivalent protocol. Included studies were evaluated using a QI classification system based on current MT QI guidelines and the Donabedian QI framework.

RESULTS

We identified 107 eligible studies. Trauma patients were the most commonly evaluated group, and total blood products transfused and in-hospital mortality were the most commonly reported QIs. Reflecting the lack of international consensus and benchmarks, we found significant variability in the reporting of QIs, which often did not reflect guideline recommendations.

DISCUSSION

Our review highlights the importance of establishing international consensus on prioritised QIs with quantifiable targets that are important to the process of MT.

Patient-Centered Care for Women: Delphi Consensus on Evidence-Derived Recommendations

OBJECTIVE

Patient-centered care (PCC) could reduce gender inequities in quality of care. Little is known about how to implement patient-centered care for women (PCCW). We aimed to generate consensus recommendations for achieving PCCW.

METHODS

We used a 2-round Delphi technique. Panelists included 21 women of varied age, ethnicity, education, and urban/rural residence; and 21 health professionals with PCC or women's health expertise. Panelists rated recommendations, derived from prior research and organized by a 6-domain PCC framework, on a 7-point Likert scale in an online survey. We used summary statistics to report response frequencies and defined consensus as when ≥85% panelists chose 5 to 7.

RESULTS

The response rate was 100%. In round 1, women and professionals retained 46 (97.9%) and 42 (89.4%) of 47 initial recommendations, respectively. The round 2 survey included 6 recommendations for women and 5 recommendations for professionals (did not achieve consensus in round 1 or were newly suggested). In round 2, women retained 2 of 6 recommendations and professionals retained 3 of 5 recommendations. Overall, 49 recommendations were generated. Both groups agreed on 44 (94.0%) recommendations (13 retained by 100% of both women and clinicians): fostering patient-physician relationship (n = 11), exchanging information (n = 10), responding to emotions (n = 4), managing uncertainty (n = 5), making decisions (n = 8), and enabling patient self-management (n = 6).

CONCLUSION

The recommendations represent the range of PCC domains, are based on evidence from primary research, and reflect high concordance between women and professional panelists. They can inform the development of policies, guidelines, programs, and performance measures that foster PCCW.

Thinking Volume First: Developing a Multifaceted Systematic Approach to Volume Management in Hemodialysis

Purpose of review:

Volume overload and hypovolemia-induced symptoms are common in the hemodialysis (HD) population and frequently result in emergency department visits and hospitalization. A structured strategy for the reporting, evaluation, and management of disordered volume status may improve clinical outcomes and the patient experience. We developed a new strategy that systematically addresses volume issues by leveraging the electronic medical record, technological adjuncts, and multidisciplinary expertise to institute new processes of care in our HD unit.

Sources of information:

This initiative was implemented in a unit located in an urban academic hospital where 250 patients receive maintenance HD. This initiative involved a multidisciplinary team of health professionals including physicians, nurse practitioners, social workers, and dieticians.

Methods:

We generated volume metrics for HD recipients based on routinely collected data from the unit’s electronic medical record. We then engaged stakeholders in a root cause analysis to identify the major causes of abnormal volume metrics locally. We subsequently developed interventions that were designed to address each of the major causes in a pragmatic and sustainable program.

Key findings:

The final product was a local volume management program with 3 components. First, we integrated volume metric reporting into the routine surveillance bloodwork reports across our unit. This enabled the clinical teams to more easily target patients at risk for volume-related adverse events and provide them with closer surveillance. Those identified with abnormal volume metrics were then evaluated with the use of technologic adjuncts such as lung ultrasound and bioimpedance spectroscopy to complement traditional assessments of volume status. Finally, those with abnormal volume metrics underwent rigorous interdisciplinary review for potential nutritional/social interventions.

Limitations:

While we report the successful initial implementation of the program within a single center, it remains unclear whether this initiative will lead to meaningful benefits for HD recipients, be readily applicable in other centers, or be sustainable in the long term.

Implications:

This volume management program will need further evaluation linked to outcome assessment and feasibility in other centers before wider adoption is advocated.

Guideline-based quality indicators-a systematic comparison of German and international clinical practice guidelines

Background

Evidence-based clinical practice guidelines (CPGs) are relevant sources for generating quality indicators (QIs). The objective of this study was to compare guideline-based QIs of German and international CPGs and their underlying methodological approaches.

Methods

We conducted systematic searches in the guideline databases of G-I-N (Guidelines International Network) and NGC (National Guideline Clearinghouse) between February and June 2017 to identify international CPGs matching the topics of German evidence-based CPGs (n = 35) that report QIs, which were identified in a preceding study. Additionally, we searched the websites of the particular CPG providers for separate documents with regard to QIs. We included evidence-based CPGs which report QIs. Reported QIs, the underlying guideline recommendations, and information on methods of development were extracted. The selection and extraction of CPGs were conducted by one reviewer and checked by another. For each matched pair of CPGs, we assessed whether the suggested QIs matched or were not directly comparable.

Results

Twenty-five international CPGs, originating from seven CPG providers in total, met the criteria for inclusion. They matched the topics of 18 German CPGs. This resulted in 30 CPG pairs for the comparison of QIs (some of the international CPGs matched the topic of more than one German CPG). We found 27 QI pairs with QIs “not different or slightly different”, corresponding to 13% (27 of 212) of the QIs in German CPGs and 16% (27 of 166) in international CPGs. Only two QI pairs were judged to be “different/inconsistent”. For 183 of 212 (86%) QIs from German CPGs and 137 of 166 (83%) QIs from international CPGs, no direct comparison could be made. An explicit link to one or more guideline recommendations was found for 136 of 152 (89%) QIs from German CPGs and 82 of 166 (49%) QIs from international CPGs. Some information on methods for the development of QIs existed for 12 of 18 (67%) German CPGs and 8 of 25 (32%) international CPGs.

Conclusions

The majority of QIs in German and international CPGs were not comparable. Various reasons for this are conceivable. More transparent reporting of the underlying methods for generating guideline-based QIs is needed.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0918-y) contains supplementary material, which is available to authorized users.

The problem with composite indicators

'The Problem with…' series covers controversial topics related to efforts to improve healthcare quality, including widely recommended but deceptively difficult strategies for improvement and pervasive problems that seem to resist solution.

National consensus recommendations on patient-centered care for ductal carcinoma in situ

PURPOSE

The purpose of this research was to generate recommendations on strategies to achieve patient-centered care (PCC) for ductal carcinoma in situ (DCIS).

METHODS

Thirty clinicians (surgeons, medical/radiation oncologists, radiologists, nurses, navigators) who manage DCIS and 32 DCIS survivors aged 18 or older were nominated. Forty-six recommendations to support PCC for DCIS were derived from primary research, and rated in a two-round Delphi process from March to June 2018.

RESULTS

A total of 29 clinicians and 27 women completed Round One, and 28 clinicians and 22 women completed Round Two. The 29 recommendations retained by both women and clinicians reflected the PCC domains of fostering patient-physician relationship (5), exchanging information (5), responding to emotions (1), managing uncertainty (4), making decisions (9), and enabling patient self-management (5). An additional 13 recommendations were retained by women only: fostering patient-physician relationship (1), exchanging information (3), responding to emotions (2), making decisions (3), and enabling patient self-management (4). Some recommendations refer to processes (i.e., ask questions about lifestyle or views about risks/outcomes to understand patient preferences); others to tools (i.e., communication aid). Panelists recommended a separate consensus process to refine the language that clinicians use when describing DCIS.

CONCLUSIONS

This is the first study to generate guidance on how to achieve PCC for DCIS. Organizations that deliver or oversee health care can use these recommendations on PCC for DCIS to plan, evaluate, or improve services. Ongoing research is needed to develop communication tools, and establish labels and language for DCIS that optimize communication.

Protocol for a mixed methods realist evaluation of regional District Health Board groupings in New Zealand

INTRODUCTION

Achieving effective integration of healthcare across primary, secondary and tertiary care is a key goal of the New Zealand (NZ) Health Strategy. NZ's regional District Health Board (DHB) groupings are fundamental to delivering integration, bringing the country's 20 DHBs together into four groups to collaboratively plan, fund and deliver health services within their defined geographical regions. This research aims to examine how, for whom and in what circumstances the regional DHB groupings work to improve health service integration, healthcare quality, health outcomes and health equity, particularly for Māori and Pacific peoples.

METHODS AND ANALYSIS

This research uses a mixed methods realist evaluation design. It comprises three linked studies: (1) formulating initial programme theory (IPT) through developing programme logic models to describe regional DHB working; (2) empirically testing IPT through both a qualitative process evaluation of regional DHB working using a case study design; and (3) a quantitative analysis of the impact that DHB regional groupings may have on service integration, health outcomes, health equity and costs. The findings of these three studies will allow refinement of the IPT and should lead to a programme theory which will explain how, for whom and in what circumstances regional DHB groupings improve service integration, health outcomes and health equity in NZ.

ETHICS AND DISSEMINATION

The University of Otago Human Ethics Committee has approved this study. The embedding of a clinician researcher within a participating regional DHB grouping has facilitated research coproduction, the research has been jointly conceived and designed and will be jointly evaluated and disseminated by researchers and practitioners. Uptake of the research findings by other key groups including policymakers, Māori providers and communities and Pacific providers and communities will be supported through key strategic relationships and dissemination activities. Academic dissemination will occur through publication and conference presentations.

Identifying positive deviants in healthcare quality and safety: a mixed methods study

Objective Solutions to quality and safety problems exist within healthcare organisations, but to maximise the learning from these positive deviants, we first need to identify them. This study explores using routinely collected, publicly available data in England to identify positively deviant services in one region of the country. Design A mixed methods study undertaken July 2014 to February 2015, employing expert discussion, consensus and statistical modelling to identify indicators of quality and safety, establish a set of criteria to inform decisions about which indicators were robust and useful measures, and whether these could be used to identify positive deviants. Setting Yorkshire and Humber, England. Participants None - analysis based on routinely collected, administrative English hospital data. Main outcome measures We identified 49 indicators of quality and safety from acute care settings across eight data sources. Twenty-six indicators did not allow comparison of quality at the sub-hospital level. Of the 23 remaining indicators, 12 met all criteria and were possible candidates for identifying positive deviants. Results Four indicators (readmission and patient reported outcomes for hip and knee surgery) offered indicators of the same service. These were selected by an expert group as the basis for statistical modelling, which supported identification of one service in Yorkshire and Humber showing a 50% positive deviation from the national average. Conclusion Relatively few indicators of quality and safety relate to a service level, making meaningful comparisons and local improvement based on the measures difficult. It was possible, however, to identify a set of indicators that provided robust measurement of the quality and safety of services providing hip and knee surgery.

Construction and pilot test of a set of indicators to assess the implementation and effectiveness of the who safe childbirth checklist

Background

The World Health Organization (WHO) launched the “Safe Childbirth Checklist (SCC) Collaboration” in 2012. The SCC is designed to contribute to quality care by providing reminders of evidence-based practices for the prevention and management of the leading causes of maternal and neonatal morbidity and mortality. However, indicators to monitor the implementation and effectiveness of the SCC have not been defined. This study aimed to produce and pilot test a set of valid, reliable and feasible indicators to assess the implementation and effectiveness of the SCC, with an emphasis on best practices.

Methods

As part of the WHO Collaboration, the SCC was adapted to the Mexican context, and a set of indicators was developed to assess the SCC use and adherence to SCC-related best practices. The indicators were pilot tested in three hospitals for feasibility and reliability using the prevalence- and bias-adjusted kappa index (PABAK) for multiple independent evaluators (initial sample, n = 47; second sample, n = 30 to re-test reliability). The data sources were clinical records and cognitive tests drawn from questionnaires to mothers and health professionals.

Results

We generated 53 indicators, and 38 of the indicators (those related to best practices and outcomes) were pilot tested. Of these, 26 relate to care for the mother (20 were measured based on clinical records and 6 via questionnaire), and 12 relate to newborn care (9 were medical record-based and 3 were from questionnaires). Feasible indicators were generally also reliable (PABAK≥0.6). Routine feasibility is affected by the frequency of assessed events.

Conclusions

The generated indicators allow an assessment of the implementation and effectiveness of the SCC and the monitoring of quality of care during childbirth and the immediate postpartum period.

Electronic supplementary material

The online version of this article (10.1186/s12884-018-1797-y) contains supplementary material, which is available to authorized users.

A Systematic Review of Quality Measures for Inflammatory Arthritis

OBJECTIVE

To conduct a systematic review and quality appraisal of quality measures for inflammatory arthritis, including rheumatoid arthritis (RA), spondyloarthritis, psoriatic arthritis (PsA), and juvenile idiopathic arthritis (JIA).

METHODS

Embase, MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) were searched from January 1, 2000, to October 23, 2016, using Medical Subject Headings terms for inflammatory arthritis and quality measures. A "grey literature" search of international arthritis organizations and quality measure libraries was also conducted. Two reviewers independently considered the papers for inclusion, with disagreements resolved by consensus. A modified guideline appraisal tool (AGREE II) was used to appraise the measure development process, which determined final inclusion. Measures were abstracted in duplicate and categorized into themes, measure type, and domains of quality.

RESULTS

Thirteen measurement sets were included from 4 countries (United States, Canada, United Kingdom, Netherlands) and 1 European consortium. They included 10 sets on RA and 1 each for PsA, inflammatory arthritis, and JIA. There were 161 unique individual measures (136 process, 20 structure, and 5 outcome). Major themes included assessment, medications, and comorbidities. Measure development methods were varied, including RAND/University of California, Los Angeles appropriateness methodology, prioritization exercises, or other modified-Delphi methods. Inclusion of patients occurred in 77% of development groups. Discussion of barriers to measurement was infrequent.

CONCLUSION

Inflammatory arthritis quality measures cover a diversity of themes encompassing process, structure, and outcomes of care across the 6 domains of quality. However, between organizations, measure development is not standardized. Local assessment of measurement feasibility before use outside the original development context is recommended.

Validation of quality indicators for end-of-life communication: results of a multicentre survey

BACKGROUND

The lack of validated quality indicators is a major barrier to improving end-of-life communication and decision-making. We sought to show the feasibility of and provide initial validation for a set of quality indicators related to end-of-life communication and decision-making.

METHODS

We administered a questionnaire to patients and their family members in 12 hospitals and asked them about advance care planning and goals-of-care discussions. Responses were used to calculate a quality indicator score. To validate this score, we determined its correlation with the concordance between the patients' expressed wishes and the medical order for life-sustaining treatments recorded in the hospital chart. We compared the correlation with concordance for the advance care planning component score with that for the goal-of-care discussion scores.

RESULTS

We enrolled 297 patients and 209 family members. At all sites, both overall quality indicators and individual domain scores were low and there was wide variability around the point estimates. The highest-ranking institution had an overall quality indicator score (95% confidence interval) of 40% (36%-44%) and the lowest had a score of 18% (11%-25%). There was a strong correlation between the overall quality indicator score and the concordance measure (r = 0.72, p = 0.008); the estimated correlation between the advance care planning score and the concordance measure (r = 0.35) was weaker than that between the goal-of-care discussion scores and the concordance measure (r = 0.53).

INTERPRETATION

Quality of end-of-life communication and decision-making appears low overall, with considerable variability across hospitals. The proposed quality indicator measure shows feasibility and partial validity. Study registration: ClinicalTrials.gov, no. NCT01362855.

Linking quality indicators to clinical trials: an automated approach

OBJECTIVE

Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations.

DESIGN

A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov.

INTERVENTION

Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity.

MAIN OUTCOME MEASURE

Number of care indicators that mapped to outcome measures in clinical trials.

RESULTS

While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method.

CONCLUSIONS

We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

Quality indicators for care of depression in primary care settings: a systematic review

BACKGROUND

Despite the growing interest in assessing the quality of care for depression, there is little evidence to support measurement of the quality of primary care for depression. This study identified evidence-based quality indicators for monitoring, evaluating and improving the quality of care for depression in primary care settings.

METHODS

Ovid MEDLINE and Ovid PsycINFO databases, and grey literature, including relevant organizational websites, were searched from 2000 to 2015. Two reviewers independently selected studies if (1) the study methodology combined a systematic literature search with assessment of quality indicators by an expert panel and (2) quality indicators were applicable to assessment of care for adults with depression in primary care settings. Included studies were appraised using the Appraisal of Indicators through Research and Evaluation (AIRE) instrument, which contains four domains and 20 items. A narrative synthesis was used to combine the indicators within themes. Quality indicators applicable to care for adults with depression in primary care settings were extracted using a structured form. The extracted quality indicators were categorized according to Donabedian's 'structure-process-outcome' framework.

RESULTS

The search revealed 3838 studies. Four additional publications were identified through grey literature searching. Thirty-nine articles were reviewed in detail and seven met the inclusion criteria. According to the AIRE domains, all studies were clear on purpose and stakeholder involvement, while formal endorsement and usage of indicators in practice were scarcely described. A total of 53 quality indicators were identified from the included studies, many of which overlap conceptually or in content: 15 structure, 33 process and four outcome indicators. This study identified quality indicators for evaluating primary care for depression among adult patients.

CONCLUSIONS

The identified set of indicators address multiple dimensions of depression care and provide an excellent starting point for further development and use in primary care settings.

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study

OBJECTIVES

Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference.

DESIGN

Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference.

SETTING

A national health information steward, the Canadian Institute for Health Information (CIHI).

PARTICIPANTS

A total of 73 participants, comprised 61 conference attendants/stakeholders from across Canada and 12 national health information steward staff.

PRIMARY AND SECONDARY OUTCOME MEASURES

Indicator dispositions of retention, additional stakeholder consultation, further redevelopment or retirement.

RESULTS

4 dimensions (usability, importance, scientific soundness and feasibility) typically used to select measures for reporting were expanded to 18 criteria grouped under the 4 dimensions through a process of research and testing. Definitions for each criterion were developed and piloted. Once the definitions were established, 56 of CIHI's publicly reported HSP indicators were evaluated against the criteria using modified Delphi approaches. Of the 56 HSP indicators evaluated, 9 measures were ratified for retirement, 7 were identified for additional consultation and 3 for further research and development. A pre-Consensus Conference survey soliciting feedback from stakeholders on indicator recommendations received 48 responses (response rate of 79%).

CONCLUSIONS

A systematic evaluation of HSP indicators informed the development of objective recommendations for continued reporting. The evaluation was a fruitful exercise to identify technical considerations for calculating indicators, furthering our understanding of how measures are used by stakeholders, as well as harmonising actions that could be taken to ensure relevancy, reduce indicator chaos and build consensus with stakeholders.

Scoping review of potential quality indicators for hip fracture patient care

OBJECTIVE

The purpose of this study is to identify existing or potential quality of care indicators (ie, current indicators as well as process and outcome measures) in the acute or postacute period, or across the continuum of care for older adults with hip fracture.

DESIGN

Scoping review.

SETTING

All care settings.

SEARCH STRATEGY

English peer-reviewed studies published from January 2000 to January 2016 were included. Literature search strategies were developed, and the search was peer-reviewed. Two reviewers independently piloted all forms, and all articles were screened in duplicate.

RESULTS

The search yielded 2729 unique articles, of which 302 articles were included (11.1%). When indicators (eg, in-hospital mortality, acute care length of stay) and potential indicators (eg, comorbidities developed in hospital, walking ability) were grouped by the outcome or process construct they were trying to measure, the most common constructs were measures of mortality (outcome), length of stay (process) and time-sensitive measures (process). There was heterogeneity in definitions within constructs between studies. There was also a paucity of indicators and potential indicators in the postacute period.

CONCLUSIONS

To improve quality of care for patients with hip fracture and create a more efficient healthcare system, mechanisms for the measurement of quality of care across the entire continuum, not just during the acute period, are required. Future research should focus on decreasing the heterogeneity in definitions of quality indicators and the development and implementation of quality indicators for the postacute period.

A retrospective cohort study of the relationship between quality indicator measurement and patient outcomes in adult trauma centers in the United States

BACKGROUND

Improving care is a key strategy for reducing the burden of injuries, but it is unknown whether the use of quality indicators (QI) is associated with patient outcomes. We sought to evaluate the association between the use of QIs by trauma centers and outcomes in adult injury patients.

METHODS

We identified consecutive adult patients (n=223,015) admitted to 233 verified trauma centers January 1, 2007 to December 31, 2010 that contributed data to the National Trauma Data Bank and participated in a survey of QI practices. Generalized Linear Mixed Models were employed to evaluate the association between the intensity (number of QIs) and nature (report cards, internal and external benchmarking) of QI use and survival to hospital discharge, adjusting for patient and hospital characteristics.

RESULTS

There were no significant differences in the odds of survival to trauma center discharge according to the number of QIs measured (quartiles; odds ratio{OR} [95% confidence interval{CI}] 1.00 vs. 1.08 [0.90-1.31] vs. 1.00 [0.82-1.22] vs. 1.21 [0.99-1.49]), or whether centers used reports cards (OR 1.07, 95%CI 0.94-1.23), internal (OR 1.06, 95%CI 0.89-1.26) or external (OR 1.09, 95%CI 0.92-1.31) benchmarking. The duration (geometric mean) of mechanical ventilation (4.0days), ICU stay (4.6days), hospital stay (7.7days) and proportion of patients with a complication (13.6%) did not significantly differ according to the intensity or nature of QI use.

CONCLUSIONS

The intensity and nature of the QIs used by trauma centers was not associated with outcomes of patient care. Alternative quality improvement strategies may be needed.

Process quality indicators for chronic kidney disease risk management: a systematic literature review

BACKGROUND AND OBJECTIVES

Quality indicators (QIs) can be used for measuring the quality of actions of healthcare providers. This systematic review gives an overview of such QIs measuring processes of care for chronic kidney disease (CKD), and identifies the QIs that have content, face, operational and/or predictive validity.

METHODS

PubMed and Embase were searched using a strategy combining the terms "quality of care," "quality indicators" and "chronic kidney disease". Papers were included if they focused on developing, testing or applying QIs for assessing the quality of care in adult patients with CKD not on renal replacement therapy.

RESULTS

Two hundred and seventy-three QIs from thirty-one papers were extracted, including QIs on adequate monitoring of kidney function and vascular risk factors, on indicated treatment, drug safety, adherence and referral to a specialist. The QIs that were considered content, face and operational valid focused on monitoring of glomerular filtration rate, albumin-creatinine ratio, lipid levels and blood pressure, the use of non-steroidal anti-inflammatory drugs, nitrofurantoin and biphosphonates in patients with CKD, and QIs on monitoring haemoglobin and treatment with angiotensin-converting-enzyme-inhibitors/angiotensin-receptor-II-blockers in patients with CKD and comorbidities. No QIs were tested for predictive validity. In addition, only two QIs focused on diet and no other QIs focused on lifestyle management.

CONCLUSIONS

Based on this review, sufficiently validated QIs can be selected for measuring the quality of CKD care. This review provides insight in QIs that need further validation, and in areas of care where QIs are still lacking.

Development and psychometric testing of a scale for the assessment of the quality of developmental care in neonatal intensive care units in Iran

Introduction

Developmental care provided to infants hospitalized at neonatal intensive care units (NICU) help weaken environmental stressors and reduce infant morbidity rates. Assessments are the first step to improving the quality of any type of care. Therefore, this study was conducted to design and assess the psychometric features of a scale designed for measuring quality of developmental care in the NICU in Iran.

Methods

This study was conducted from December 2014 through September 2015 in Tehran, Iran. The present mixed-methods sequential exploratory (quantitative-qualitative) study used the Delphi method to design an initial questionnaire through a review of the literature and by using the input of experts. The validity of the questionnaire was ensured by assessing then validity of its content (qualitative-quantitative), face (qualitative-quantitative), and construct (exploratory factor analysis with 500 NICU personnel from 34 hospitals in Tehran), and its reliability was ensured by assessing its internal consistency (using Cronbach’s alpha) and by assessing its stability through the test-retest method.

Results

The qualitative stage of the study resulted in a 93-item questionnaire with eight domains. After performing the content and face analyses, a factor analysis was performed on 90 items of the questionnaire, yielding a 76-item questionnaire with five domains, including “sleep, pain and stress management,” “routine care,” “the family,” “management,” and “sensory care,” which explained 62.5% of the variance. The reliability of the questionnaire was confirmed with a Cronbach’s alpha of 0.9 and its stability was confirmed by an Intraclass Correlation Coefficient (ICC) of 0.93.

Conclusion

The questionnaire developed for the assessment of developmental care in the NICU covered all of the dimensions of this type of care, and it is a valid and reliable tool for assessing and improving developmental care in the NICU.

Quality and performance measures of strain on intensive care capacity: a protocol for a systematic review

BACKGROUND

The matching of critical care service supply with demand is fundamental for the efficient delivery of advanced life support to patients in urgent need. Mismatch in this supply/demand relationship contributes to "intensive care unit (ICU) capacity strain," defined as a time-varying disruption in the ability of an ICU to provide well-timed and high-quality intensive care support to any and all patients who are or may become critically ill. ICU capacity strain leads to suboptimal quality of care and may directly contribute to heightened risk of adverse events, premature discharges, unplanned readmissions, and avoidable death. Unrelenting strain on ICU capacity contributes to inefficient health resource utilization and may negatively impact the satisfaction of patients, their families, and frontline providers. It is unknown how to optimally quantify the instantaneous and temporal "stress" an ICU experiences due to capacity strain.

METHODS

We will perform a systematic review to identify, appraise, and evaluate quality and performance measures of strain on ICU capacity and their association with relevant patient-centered, ICU-level, and health system-level outcomes. Electronic databases (i.e., MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Web of Science, and the Agency of Healthcare Research and Quality (AHRQ) - National Quality Measures Clearinghouse (NQMC)) will be searched for original studies of measures of ICU capacity strain. Selected gray literature sources will be searched. Search themes will focus on intensive care, quality, operations management, and capacity. Analysis will be primarily narrative. Each identified measure will be defined, characterized, and evaluated using the criteria proposed by the US Strategic Framework Board for a National Quality Measurement and Reporting System (i.e., importance, scientific acceptability, usability, feasibility).

DISCUSSION

Our systematic review will comprehensively identify, define, and evaluate quality and performance measures of ICU capacity strain. This is a necessary step towards understanding the impact of capacity strain on quality and performance in intensive care and to develop innovative interventions aimed to improve efficiency, avoid waste, and better anticipate impending capacity shortfalls.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO, CRD42015017931.

Developing 'high impact' guideline-based quality indicators for UK primary care: a multi-stage consensus process

BACKGROUND

Quality indicators (QIs) are an important tool for improving clinical practice and are increasingly being developed from evidence-based guideline recommendations. We aimed to identify, select and apply guideline recommendations to develop a set of QIs to measure the implementation of evidence-based practice using routinely recorded clinical data in United Kingdom (UK) primary care.

METHODS

We reviewed existing national clinical guidelines and QIs and used a four-stage consensus development process to derive a set of 'high impact' QIs relevant to primary care based upon explicit prioritisation criteria. We then field tested the QIs using remotely extracted, anonymised patient records from 89 randomly sampled primary care practices in the Yorkshire region of England.

RESULTS

Out of 2365 recommendations and QIs originally reviewed, we derived a set of 18 QIs (5 single, 13 composites - comprising 2-9 individual recommendations) for field testing. QIs predominantly addressed chronic disease management, in particular diabetes, cardiovascular and renal disease, and included both processes and outcomes of care. Field testing proved to be critical for further refinement and final selection.

CONCLUSIONS

We have demonstrated a rigorous and transparent methodology to develop a set of high impact, evidence-based QIs for primary care from clinical guideline recommendations. While the development process was successful in developing a limited set of QIs, it remains challenging to derive robust new QIs from clinical guidelines in the absence of established systems for routine, structured recording of clinical care.

What Aspects of Personal Care Are Most Important to Patients Undergoing Radiation Therapy for Prostate Cancer?

PURPOSE/OBJECTIVE

The overall quality of patient care is a function of the quality of both its technical and its nontechnical components. The purpose of this study was to identify the elements of nontechnical (personal) care that are most important to patients undergoing radiation therapy for prostate cancer.

METHODS AND MATERIALS

We reviewed the literature and interviewed patients and health professionals to identify elements of personal care pertinent to patients undergoing radiation therapy for prostate cancer. We identified 143 individual elements relating to 10 aspects of personal care. Patients undergoing radical radiation therapy for prostate cancer completed a self-administered questionnaire in which they rated the importance of each element. The overall importance of each element was measured by the percentage of respondents who rated it as "very important." The importance of each aspect of personal care was measured by the mean importance of its elements.

RESULTS

One hundred eight patients completed the questionnaire. The percentage of patients who rated each element "very important" ranged from 7% to 95% (mean 61%). The mean importance rating of the elements of each aspect of care varied significantly: "perceived competence of caregivers," 80%; "empathy and respectfulness of caregivers," 67%; "adequacy of information sharing," 67%; "patient centeredness," 59%; "accessibility of caregivers," 57%; "continuity of care," 51%; "privacy," 51%; "convenience," 45%; "comprehensiveness of services," 44%; and "treatment environment," 30% (P<.0001). Neither age nor education was associated with importance ratings, but the patient's health status was associated with the rating of some elements of care.

CONCLUSIONS

Many different elements of personal care are important to patients undergoing radiation therapy for prostate cancer, but the 3 aspects of care that most believe are most important are these: the perceived competence of their caregivers, the empathy and respectfulness of their caregivers, and the adequacy of information sharing.

Comparing clinical quality indicators for asthma management in children with outcome measures used in randomised controlled trials: a protocol

INTRODUCTION

Clinical quality indicators are necessary to monitor the performance of healthcare services. The development of indicators should, wherever possible, be based on research evidence to minimise the risk of bias which may be introduced during their development, because of logistic, ethical or financial constraints alone. The development of automated methods to identify the evidence base for candidate indicators should improve the process of indicator development. The objective of this study is to explore the relationship between clinical quality indicators for asthma management in children with outcome and process measurements extracted from randomised controlled clinical trial reports.

METHODS AND ANALYSIS

National-level indicators for asthma management in children will be extracted from the National Quality Measures Clearinghouse (NQMC) database and the National Institute for Health and Care Excellence (NICE) quality standards. Outcome measures will be extracted from published English language randomised controlled trial (RCT) reports for asthma management in children aged below 12 years. The two sets of measures will be compared to assess any overlap. The study will provide insights into the relationship between clinical quality indicators and measurements in RCTs. This study will also yield a list of measurements used in RCTs for asthma management in children, and will find RCT evidence for indicators used in practice.

ETHICS AND DISSEMINATION

Ethical approval is not necessary because this study will not include patient data. Findings will be disseminated through peer-reviewed publications.

Improving End-of-Life Communication and Decision Making: The Development of a Conceptual Framework and Quality Indicators

CONTEXT

The goal of end-of-life (EOL) communication and decision making is to create a shared understanding about a person's values and treatment preferences that will lead to a plan of care that is consistent with these values and preferences. Improvements in communication and decision making at the EOL have been identified as a high priority from a patient and family point of view.

OBJECTIVES

The purpose of this study was to develop quality indicators related to EOL communication and decision making.

METHODS

We convened a multidisciplinary panel of experts to develop definitions, a conceptual framework of EOL communication and decision making, and quality indicators using a modified Delphi method. We generated a list of potential items based on literature review and input from panel members. Panel members rated the items using a seven-point Likert scale (1 = very little importance to 7 = extremely important) over four rounds of review until consensus was achieved.

RESULTS

About 24 of the 28 panel members participated in all four rounds of the Delphi process. The final list of quality indicators comprised 34 items, divided into the four categories of our conceptual framework: Advance care planning (eight items), Goals of care discussions (13 items), Documentation (five items), and Organization/System aspects (eight items). Eleven items were rated "extremely important" (median score). All items had a median score of five (moderately important) or greater.

CONCLUSION

We have developed definitions, a conceptual framework, and quality indicators that researchers and health care decision makers can use to evaluate and improve the quality of EOL communication and decision making.

Quality indicators for hip fracture patients: a scoping review protocol

INTRODUCTION

Hip fractures are a significant cause of morbidity and mortality and care of hip fracture patients places a heavy burden on healthcare systems due to prolonged recovery time. Measuring quality of care delivered to hip fracture patients is important to help target efforts to improve care for patients and efficiency of the health system. The purpose of this study is to synthesise the evidence surrounding quality of care indicators for patients who have sustained a hip fracture. Using a scoping review methodology, the research question that will be addressed is: "What patient, institutional, and system-level indicators are currently in use or proposed for measuring quality of care across the continuum for individuals following a hip fracture?".

METHODS AND ANALYSIS

We will employ the methodological frameworks used by Arksey and O'Malley and Levac et al. The synthesis will be limited to quality of care indicators for individuals who suffered low trauma hip fracture. All English peer-reviewed studies published from the year 2000-most recent will be included. Literature search strategies will be developed using medical subject headings and text words related to hip fracture quality indicators and the search will be peer-reviewed. Numerous electronic databases will be searched. Two reviewers will independently screen titles and abstracts for inclusion, followed by screening of the full text of potentially relevant articles to determine final inclusion. Abstracted data will include study characteristics and indicator definitions.

DISSEMINATION

To improve quality of care for patients and create a more efficient healthcare system, mechanisms for the measurement of quality of care are required. The implementation of quality of care indicators enables stakeholders to target areas for improvement in service delivery. Knowledge translation activities will occur throughout the review with dissemination of the project goals and findings to local, national, and international stakeholders.