Evaluating quality indicators for physical therapy in primary care

Developing an initial set of quality indicators for chiropractic care: a scoping review

BACKGROUND

Quality indicators are standardized, evidence-based measures of health care quality. Currently, there is no basic set of quality indicators for chiropractic care published in peer-reviewed literature. The goal of this research is to develop a preliminary set of quality indicators, measurable with administrative data.

METHODS

We conducted a scoping review searching PubMed/MEDLINE, CINAHL, and Index to Chiropractic Literature databases. Eligible articles were published after 2011, in English, developing/reporting best practices and clinical guidelines specifically developed for, or directly applicable to, chiropractic care. Eligible non-peer-reviewed sources such as quality measures published by the Centers for Medicare and Medicaid Services and the Royal College of Chiropractors quality standards were also included. Following a stepwise eligibility determination process, data abstraction identified specific statements from included sources that can conceivably be measured with administrative data. Once identified, statements were transformed into potential indicators by: 1) Generating a brief title and description; 2) Documenting a source; 3) Developing a metric; and 4) Assigning a Donabedian category (structure, process, outcome). Draft indicators then traversed a 5-step assessment: 1) Describes a narrowly defined structure, process, or outcome; 2) Quantitative data can conceivably be available; 3) Performance is achievable; 4) Metric is relevant; 5) Data are obtainable within reasonable time limits. Indicators meeting all criteria were included in the final set.

RESULTS

Literature searching revealed 2562 articles. After removing duplicates and conducting eligibility determination, 18 remained. Most were clinical guidelines (n = 10) and best practice recommendations (n = 6), with 1 consensus and 1 clinical standards development study. Data abstraction and transformation produced 204 draft quality indicators. Of those, 57 did not meet 1 or more assessment criteria. After removing duplicates, 70 distinct indicators remained. Most indicators matched the Donabedian category of process (n = 35), with 31 structure and 4 outcome indicators. No sources were identified to support indicator development from patient perspectives.

CONCLUSIONS

This article proposes a preliminary set of 70 quality indicators for chiropractic care, theoretically measurable with administrative data and largely obtained from electronic health records. Future research should assess feasibility, achieve stakeholder consensus, develop additional indicators including those considering patient perspectives, and study relationships with clinical outcomes.

TRIAL REGISTRATION

Open Science Framework, https://osf.io/t7kgm.

Improved quality of physiotherapy care in patients with Whiplash-Associated Disorders: Results based on 16 years of routinely collected data

Quality improvement is now a central tenet in physiotherapy care, and quality indicators (QIs), as measurable elements of care, have been applied to analyze and evaluate the quality of physiotherapy care over the past two decades. QIs, based on Donabedian's model of quality of care, provide a foundation for measuring (improvements in) quality of physiotherapy care, providing insight into the many remaining evidentiary gaps concerning diagnostics, prognostics and treatment, as well as patient-related outcome measures. In this overview we provide a synthesis of four recently published articles from our project group on the topic of quantitative measures of quality improvement in physiotherapy care, in this context specifically focused on patients with WAD in primary care physiotherapy. A set of process and outcome QIs (n = 28) was developed for patients with WAD and linked to a database consisting of routinely collected data (RCD) on patients with WAD collected over a 16-year period. The QIs were then embedded per step of the clinical reasoning process: (a) administration (n = 2); (b) history taking (n = 7); (c) objectives of examination (n = 1); (d) clinical examination (n = 5); (e) analysis and conclusion (n = 1); (f) treatment plan (n = 3); (g) treatment (n = 2); (h) evaluation (n = 5); and (i) discharge (n = 2). QIs were expressed as percentages, allowing target performance levels to be defined ≥70% or ≤30%, depending on whether the desired performance required an initially high or low QI score. Using RCD data on primary care patients with WAD (N = 810) and a set of QIs, we found that the quality of physiotherapy care has improved substantially over a 16-year period. This conclusion was based on QIs meeting predetermined performance targets of ≥70% or ≤30%. Twenty-three indicators met the target criterium of ≥70% and three indicators ≤30%. Our recommended set of QIs, embedded in a clinical reasoning process for patients with WAD, can now be used as a basis for the development of a validated QI set that effectively measures quality (improvement) of primary care physiotherapy in patients with WAD.

A practice test and selection of a core set of outcome-based quality indicators in Dutch primary care physical therapy for patients with COPD: a cohort study

Aim

To estimate the comparability and discriminability of outcome-based quality indicators by performing a practice test in Dutch physical therapy primary care, and to select a core set of outcome-based quality indicators that are well accepted by physical therapists based on their perceived added value as a quality improvement tool.

Methods

First, a list of potential quality indicators was defined, followed by determination of the comparability (case-mix adjusted multilevel analysis) and discriminability (intraclass correlation coefficient (ICC)). Second, focus group meetings were conducted with stakeholders (physical therapists and senior researchers) to select a core set of quality indicators.

Results

Overall, 229 physical therapists from 137 practices provided 2651 treatment episodes. Comparability: in 10 of the 11 case-mix adjusted models, the ICC increased compared with the intercept-only model. Discriminability: the ICC ranged between 0.01 and 0.34, with five of the 11 ICCs being >0.10. The majority of physical therapists in each focus group preferred the inclusion of seven quality indicators in the core set, including three process and four outcome indicators based upon the 6-min walk test (6MWT), the Clinical COPD Questionnaire (CCQ), and the determination of quadriceps strength using a hand-held dynamometer.

Conclusion

This is the first study to describe the comparability and discriminability of the outcome-based quality indicators selected for patients with COPD treated in primary care physical therapy practices. Future research should focus on increasing data collection in daily practice and on the development of tangible methods to use as the core set of a quality improvement tool.

The major finding of this study is that all participants in the focus groups accepted the quality indicators as a quality improvement tool based on their perceived added value and selected a core set of seven outcome-based quality indicators for COPD.https://bit.ly/3NJuQtq

Patient-Reported Outcome-Based Quality Indicators in Dutch Primary Care Physical Therapy for Patients With Nonspecific Low Back Pain: A Cohort Study

OBJECTIVE

The purpose of this study was to define and select a core set of outcome-based quality indicators, accepted by stakeholders on usability and perceived added value as a quality improvement tool, and to formulate recommendations for the next implementation step.

METHODS

In phase 1, 15 potential quality indicators were defined for patient-reported outcome measures and associated domains, namely the Numeric Pain Rating Scale (NPRS) for pain intensity, the Patient Specific Functioning Scale (PSFS) for physical activity, the Quebec Back Pain Disability Scale for physical functioning, and the Global Perceived Effect-Dutch Version for perceived effect. Their comparability and discriminatory characteristics were described using cohort data. In phase 2, a core set of quality indicators was selected based on consensus among stakeholders in focus group meetings.

RESULTS

In total, 65,815 completed treatment episodes for patients with nonspecific low back pain were provided by 1009 physical therapists from 219 physical therapist practices. The discriminability between physical therapists of all potential 15 quality indicators was adequate, with intraclass correlation coefficients between 0.08 and 0.30. Stakeholders selected a final core set of 6 quality indicators: 2 process indicators (the routine measurement of NPRS and the PSFS) and 4 outcome indicators (pretreatment and posttreatment change scores for the NPRS, PSFS, Quebec Back Pain Disability Scale, and the minimal clinically important difference of the Global Perceived Effect-Dutch Version).

CONCLUSION

This study described and selected a core set of outcome-based quality indicators for physical therapy in patients with nonspecific low back pain. The set was accepted by stakeholders for having added value for daily practice in physical therapy primary care and was found useful for quality improvement initiatives. Further studies need to focus on improvement of using the core set of outcome-based quality indicators as a quality monitoring and evaluation instrument.

IMPACT

Patient-reported outcome-based quality indicators developed from routinely collected clinical data are promising for use in quality improvement in daily practice.

Relationships Between Context, Process, and Outcome Indicators to Assess Quality of Physiotherapy Care in Patients with Whiplash-Associated Disorders: Applying Donabedian's Model of Care

BACKGROUND

Quality indicators (QIs) are measurable elements of practice performance and may relate to context, process, outcome and structure. A valid set of QIs have been developed, reflecting the clinical reasoning used in primary care physiotherapy for patients with whiplash-associated disorders (WAD). Donabedian's model postulates relationships between the constructs of quality of care, acting in a virtuous circle.

AIM

To explore the relative strengths of the relationships between context, process, and outcome indicators in the assessment of primary care physiotherapy in patients with WAD.

MATERIALS AND METHODS

Data on WAD patients (N=810) were collected over a period of 16 years in primary care physiotherapy practices by means of patients records. This routinely collected dataset (RCD-WAD) was classified in context, process, and outcome variables and analyzed retrospectively. Clinically relevant variables were selected based on expert consensus. Associations were expressed, using zero-order, as Spearman rank correlation coefficients (criterion: rs >0.25 [minimum: fair]; α-value = 0.05).

RESULTS

In round 1, 62 of 85 (72.9%) variables were selected by an expert panel as relevant for clinical reasoning; in round 2, 34 of 62 (54.8%) (context variables 9 of 18 [50.0%]; process variables 18 of 34 [52.9]; outcome variables 8 of 10 [90.0%]) as highly relevant. Associations between the selected context and process variables ranged from 0.27 to 0.53 (p≤0.00), between selected context and outcome variables from 0.26 to 0.55 (p≤0.00), and between selected process and outcome variables from 0.29 to 0.59 (p≤0.00). Moderate associations (rs >0.50; p≤0.00) were found between "pain coping" and "fear avoidance" as process variables, and "pain intensity" and "functioning" as outcome variables.

CONCLUSION

The identified associations between selected context, process, and outcome variables were fair to moderate. Ongoing work may clarify some of these associations and provide guidance to physiotherapists on how best to improve the quality of clinical reasoning in terms of relationships between context, process, and outcome in the management of patients with WAD.

Has the quality of physiotherapy care in patients with Whiplash-associated disorders (WAD) improved over time? A retrospective study using routinely collected data and quality indicators

PURPOSE

To develop valid quality indicators (QIs) for physiotherapy care based on best available evidence, and to use these QIs to explore trends in the quality of physiotherapy care of patients with Whiplash-associated disorders (WAD) using guideline-based routinely collected data (RCD) gathered between 1996 and 2011.

MATERIALS AND METHODS

The study consisted of two phases: 1) development of QIs and 2) analysis of patient records. A set of QIs was developed based on recommendations in the scientific literature and the Dutch Clinical Practice Guideline (CPG) "Physiotherapy Management and WAD". QIs were expressed as percentages, allowing target performance levels to be defined (≥80% or ≤30% depending on whether desired performance required a high or low score on a QI). We then analyzed WAD patient data (N = 810) collected over a period of 16 years in two physiotherapy practices, separating patients into two groups defined as before (Group A 1996-2002; n = 353) and after (Group B 2003-2011; n = 457) implementation and transition to the Dutch CPG "Physiotherapy Management and WAD".

RESULTS

Using an iterative process and input from both experts and users, 28 QIs were developed and subsequently classified per step of the clinical reasoning process for physiotherapy care. Based on 16 years of RCD, we found that the clinical reasoning process differed significantly (P ≤ 0.05) between the groups, in favor of Group B. Twelve of the 25 indicators (48.0%) in Group A and 19 of 26 indicators (73.1%) in Group B met predetermined performance targets. The number of target indicators also differed significantly between groups, favoring Group B (P ≤ 0.05).

CONCLUSION

A preliminary set of novel QIs was developed. Using RCD and these QIs, we conclude that physiotherapy care in our study setting improved over the period 1996-2011. Furthermore, the QIs met the performance targets set for the clinical reasoning process after the transition to the Dutch CPG "Physiotherapy Management and WAD".

Pay-for-performance reduces healthcare spending and improves quality of care: Analysis of target and non-target obstetrics and gynecology surgeries

Objective

In Korea, the Value Incentive Program (VIP) was first applied to selected clinical conditions in 2007 to evaluate the performance of medical institutes. We examined whether the condition-specific performance of the VIP resulted in measurable improvement in quality of care and in reduced medical costs.

Design

Population-based retrospective observational study.

Setting

We used two data set including the results of quality assessment and hospitalization data from National Health Claim data from 2011 to 2014.

Participants

Participants who were admitted to the hospital for obstetrics and gynecology were included. A total of 535 289 hospitalizations were included in our analysis.

Methods

We used a generalized estimating equation (GEE) model to identify associations between the quality assessment and length of stay (LOS). A GEE model based on a gamma distribution was used to evaluate medical cost. The Poisson regression analysis was used to evaluate readmission.

Main Outcome Measures

The outcome variables included LOS, medical costs and readmission within 30 days.

Results

Higher condition-specific performance by VIP participants was associated with shorter LOSs, decreases in medical cost, and lower within 30-day readmission rates for target and non-target surgeries. LOS and readmission within 30 days were different by change in quality assessment at each medical institute.

Conclusions

Our findings contribute to the body of evidence used by policy-makers for expansion and development of the VIP. The study revealed the positive effects of quality assessment on quality of care. To reduce the between-institute quality gap, alternative strategies are needed for medical institutes that had low performance.

Data extraction from electronic health records (EHRs) for quality measurement of the physical therapy process: comparison between EHR data and survey data

BACKGROUND

With the emergence of the electronic health records (EHRs) as a pervasive healthcare information technology, new opportunities and challenges for use of clinical data for quality measurements arise with respect to data quality, data availability and comparability. The objective of this study is to test whether data extracted from electronic health records (EHRs) was of comparable quality as survey data for the calculation of quality indicators.

METHODS

Data from surveys describing patient cases and filled out by physiotherapists in 2009-2010 were used to calculate scores on eight quality indicators (QIs) to measure the quality of physiotherapy care. In 2011, data was extracted directly from EHRs. The data collection methods were evaluated for comparability. EHR data was compared to survey data on completeness and correctness.

RESULTS

Five of the eight QIs could be extracted from the EHRs. Three were omitted from the indicator set, as they proved too difficult to be extracted from the EHRs. Another QI proved incomparable due to errors in the extraction software of some of the EHRs. Three out of four comparable QIs performed better (p < 0.001) in EHR data on completeness. EHR data also proved to be correct; the relative change in indicator scores between EHR and survey data were small (<5 %) in three out of four QIs.

CONCLUSION

Data quality of EHRs was sufficient to be used for the calculation of QIs, although comparability to survey data was problematic. Standardization is needed, not only to be able to compare different data collection methods properly, but also to compare between practices with different EHRs. EHRs have the option to administrate narrative data, but natural language processing tools are needed to quantify these text boxes. Such development, can narrow the comparability gap between scoring QIs based on EHR data and based on survey data. EHRs have the potential to provide real time feedback to professionals and quality measurements for research, but more effort is needed to create unambiguous and uniform information and to unlock written text in a standardized manner.

The Reasons behind the (Non)Use of Feedback Reports for Quality Improvement in Physical Therapy: A Mixed-Method Study

OBJECTIVES

To explain the use of feedback reports for quality improvements by the reasons to participate in quality measuring projects and to identify barriers and facilitators.

DESIGN

Mixed methods design.

METHODS

In 2009-2011 a national audit and feedback system for physical therapy (Qualiphy) was initiated in the Netherlands. After each data collection round, an evaluation survey was held amongst its participants. The evaluation survey data was used to explain the use of feedback reports by studying the reasons to participate with Qualiphy with correlation measures and logistic regression. Semi-structured interviews with PTs served to seek confirmation and disentangle barriers and facilitators.

RESULTS

Analysis of 257 surveys (response rate: 42.8%) showed that therapists with only financial reasons were less likely to use feedback reports (OR = 0.24;95%CI = 0.11-0.52) compared to therapists with a mixture of reasons. PTs in 2009 and 2010 were more likely to use the feedback reports for quality improvement than PTs in 2011 (OR = 2.41;95%CI = 1.25-4.64 respectively OR = 3.28;95%CI = 1.51-7.10). Changing circumstances in 2011, i.e. using EHRs and financial incentives, had a negative effect on the use of feedback reports (OR = 0.40, 95%CI = 0.20-0.78). Interviews with 12 physical therapists showed that feedback reports could serve as a tool to support and structure quality improvement plans. Barriers were distrust and perceived self-reporting bias on indicator scores.

CONCLUSIONS

Implementing financial incentives that are not well-specified and well-targeted can have an adverse effect on using feedback reports to improve quality of care. Distrust is a major barrier to implementing quality systems.

Quality of care in primary health care settings in the Eastern Mediterranean region: a systematic review of the literature

PURPOSE

This systematic review aims at offering a comprehensive synthesis of studies addressing quality of care in the primary healthcare (PHC) sector of the Eastern Mediterranean Region (EMR).

DATA SOURCES

A systematic search was conducted using Medline, Embase and Global Health Library (IMEMR) electronic databases to identify studies related to quality in PHC between years 2000 and 2012.

STUDY SELECTION/DATA EXTRACTION

One hundred and fifty-nine (159) studies fulfilled the eligibility criteria. Each paper was independently reviewed by two reviewers, and the following information was extracted/calculated: dimension of care investigated (structure, processes and outcomes), focus, disease groups, study design, sample size, unit of analysis, response rate, country, setting (public or private) and level of rigor (LOR) score.

RESULTS OF DATA SYNTHESIS

Most of the studies were descriptive/cross-sectional in nature with a relatively modest LOR score. Assessment of quality of care revealed that the process dimension of quality, specifically clinical practice and patient-provider relationship, is an area of major concern. However, interventions targeting enhanced quality in PHC in the EMR countries had favorable and effective outcomes in terms of clinical practice.

CONCLUSION

These findings highlight gaps in evidence on quality in PHC in the EMR; such evidence is key for decision-making. Researchers and policy-makers should address these gaps to generate contextualized information and knowledge that ensures relevance and targeted high-impact interventions.