Patients' and clinicians' views of comparing the performance of providers of surgery: a qualitative study

A practice test and selection of a core set of outcome-based quality indicators in Dutch primary care physical therapy for patients with COPD: a cohort study

Aim

To estimate the comparability and discriminability of outcome-based quality indicators by performing a practice test in Dutch physical therapy primary care, and to select a core set of outcome-based quality indicators that are well accepted by physical therapists based on their perceived added value as a quality improvement tool.

Methods

First, a list of potential quality indicators was defined, followed by determination of the comparability (case-mix adjusted multilevel analysis) and discriminability (intraclass correlation coefficient (ICC)). Second, focus group meetings were conducted with stakeholders (physical therapists and senior researchers) to select a core set of quality indicators.

Results

Overall, 229 physical therapists from 137 practices provided 2651 treatment episodes. Comparability: in 10 of the 11 case-mix adjusted models, the ICC increased compared with the intercept-only model. Discriminability: the ICC ranged between 0.01 and 0.34, with five of the 11 ICCs being >0.10. The majority of physical therapists in each focus group preferred the inclusion of seven quality indicators in the core set, including three process and four outcome indicators based upon the 6-min walk test (6MWT), the Clinical COPD Questionnaire (CCQ), and the determination of quadriceps strength using a hand-held dynamometer.

Conclusion

This is the first study to describe the comparability and discriminability of the outcome-based quality indicators selected for patients with COPD treated in primary care physical therapy practices. Future research should focus on increasing data collection in daily practice and on the development of tangible methods to use as the core set of a quality improvement tool.

The major finding of this study is that all participants in the focus groups accepted the quality indicators as a quality improvement tool based on their perceived added value and selected a core set of seven outcome-based quality indicators for COPD.https://bit.ly/3NJuQtq

An Evidence-Based Theory About PRO Use in Kidney Care: A Realist Synthesis

BACKGROUND

There is international interest on the use of patient-reported outcomes (PROs) in nephrology.

OBJECTIVES

Our objectives were to develop a kidney-specific program theory about use of PROs in nephrology that may enhance person-centered care, both at individual and aggregated levels of care, and to test and refine this theory through a systematic review of the empirical literature. Together, these objectives articulate what works or does not work, for whom, and why.

METHODS

Realist synthesis methodology guided the electronic database and gray literature searches (in January 2017 and October 2018), screening, and extraction conducted independently by three reviewers. Sources included all nephrology patients and/or practitioners. Through a process of extraction and synthesis, each included source was examined to assess how contexts may trigger mechanisms to influence specific outcomes.

RESULTS

After screening 19,961 references, 84 theoretical and 34 empirical sources were used. PROs are proposed to be useful for providing nephrology care through three types of use. The first type is use of individual-level PRO data at point of care, receiving the majority of theoretical and empirical explorations. Clinician use to support person-centered care, and patient use to support patient engagement, are purported to improve satisfaction, health, and quality of life. Contextual factors specific to the kidney care setting that may influence the use of PRO data include the complexity of kidney disease symptom burden, symptoms that may be stigmatized, comorbidities, and time or administrative constraints in dialysis settings. Electronic collection of PROs may facilitate PRO use given these contexts. The second type is use of aggregated PRO data at point of care, including public reporting of PROs to inform decisions at point of care and improve quality of care, and use of PROs for treatment decisions. The third type is use of aggregated PRO data by organizations, including publicly available PRO data to compare centers. In single-payer systems, regular collection of PROs by dialysis centers can be achieved through economic incentives. Both the second and third types of PRO use include pressures that may trigger quality improvement processes.

CONCLUSION

The current state of the evidence is primarily theoretical. There is pressing need for empirical research to improve the evidence-base of PRO use at individual and aggregated levels of nephrology care.

Patient-Reported Outcome-Based Quality Indicators in Dutch Primary Care Physical Therapy for Patients With Nonspecific Low Back Pain: A Cohort Study

OBJECTIVE

The purpose of this study was to define and select a core set of outcome-based quality indicators, accepted by stakeholders on usability and perceived added value as a quality improvement tool, and to formulate recommendations for the next implementation step.

METHODS

In phase 1, 15 potential quality indicators were defined for patient-reported outcome measures and associated domains, namely the Numeric Pain Rating Scale (NPRS) for pain intensity, the Patient Specific Functioning Scale (PSFS) for physical activity, the Quebec Back Pain Disability Scale for physical functioning, and the Global Perceived Effect-Dutch Version for perceived effect. Their comparability and discriminatory characteristics were described using cohort data. In phase 2, a core set of quality indicators was selected based on consensus among stakeholders in focus group meetings.

RESULTS

In total, 65,815 completed treatment episodes for patients with nonspecific low back pain were provided by 1009 physical therapists from 219 physical therapist practices. The discriminability between physical therapists of all potential 15 quality indicators was adequate, with intraclass correlation coefficients between 0.08 and 0.30. Stakeholders selected a final core set of 6 quality indicators: 2 process indicators (the routine measurement of NPRS and the PSFS) and 4 outcome indicators (pretreatment and posttreatment change scores for the NPRS, PSFS, Quebec Back Pain Disability Scale, and the minimal clinically important difference of the Global Perceived Effect-Dutch Version).

CONCLUSION

This study described and selected a core set of outcome-based quality indicators for physical therapy in patients with nonspecific low back pain. The set was accepted by stakeholders for having added value for daily practice in physical therapy primary care and was found useful for quality improvement initiatives. Further studies need to focus on improvement of using the core set of outcome-based quality indicators as a quality monitoring and evaluation instrument.

IMPACT

Patient-reported outcome-based quality indicators developed from routinely collected clinical data are promising for use in quality improvement in daily practice.

Determinants of physician networks: an ethnographic study examining the processes that inform patterns of collaboration and referral decision-making among physicians

OBJECTIVE

Most scholarly attention to studying collaborative ties in physician networks has been devoted to quantitatively analysing large, complex datasets. While valuable, such studies can reduce the dynamic and contextual complexities of physician collaborations to numerical values. Qualitative research strategies can contribute to our understanding by addressing the gaps left by more quantitative approaches. This study seeks to contribute to the literature that applies network science approaches to the context of healthcare delivery. We use qualitative, observational and interview, methods to pursue an in-depth, micro-level approach to the deeply social and discursive processes that influence patterns of collaboration and referral decision-making in physician networks.

DESIGN

Qualitative methodologies that paired ethnographic field observations, semistructured interviews and document analysis were used. An inductive thematic analysis approach was used to analyse, identify and describe patterns in those data.

SETTING

This study took place in a high-volume cardiovascular department at a major academic medical centre (AMC) located in the Midwest region of the USA.

PARTICIPANTS

Purposive and snowballing sampling were used to recruit study participants for both the observational and face-to-face in-depth interview portions of the study. In total, 25 clinicians and 43 patients participated in this study.

RESULTS

Two primary thematic categories were identified: (1) circumstances for external engagement; and (2) clinical conditions for engagement. Thematic subcategories included community engagement, scientific engagement, reputational value, experiential information, professional identity, self-awareness of competence, multidisciplinary programmes and situational factors.

CONCLUSION

This study adds new contextual knowledge about the mechanisms that characterise referral decision-making processes and how these impact the meaning of physician relationships, organisation of healthcare delivery and the knowledge and beliefs that physicians have about their colleagues. This study highlights the nuances that influence how new collaborative networks are formed and maintained by detailing how relationships among physicians develop and evolve over time.

Patient Views on Surgeon-specific Outcome Reporting in Vascular Surgery: Novel Validated Patient Questionnaire Study

BACKGROUND

SSMD are used to enhance transparency, improve quality and facilitate patient choice. The use of SSMD is controversial, but patients' views on such data are largely unknown.

OBJECTIVES

The aim of this study was therefore to explore the views of patients and to identify their priorities for outcome reporting in vascular surgery.

METHODS

A prospective questionnaire study of 165 patients receiving care in a single academic vascular unit was performed. Data on patients' current understanding and use of SSMD, together with future priorities were collected.

RESULTS

Of the 165 patients 80% were unaware of SSMD. 72% thought they should be made aware of the data, although 63% thought they were likely to misinterpret the results. The majority recognized the utility of SSMD to inform treatment (60%) and surgeon (53%) choice. The majority prioritize the patient-surgeon relationship (90%) and past experiences of care (71%) when making treatment decisions. A significant majority (66% vs 49%; P < 0.005) would favour hospital-level to surgeon-level data. The main patient priorities for future outcome reporting were waiting list length (56%), the quality of hospital facilities (55%), and patient satisfaction (54%).

CONCLUSIONS

The aims of SSMD reporting are not currently being met, and both patients and healthcare professionals have shared concerns over the nature and usefulness of the data. Patients express a preference for hospital-level outcomes and prioritize the experience of receiving care over outcomes when making treatment decisions. Future outcome reporting should include patient-directed hospital-level metrics that are readily accessible and understood by all.

How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis

Objectives Internationally, there has been considerable debate about the role of data in supporting quality improvement in health care. Our objective was to understand how, why and in what circumstances the feedback of aggregated patient-reported outcome measures data improved patient care. Methods We conducted a realist synthesis. We identified three main programme theories underlying the use of patient-reported outcome measures as a quality improvement strategy and expressed them as nine 'if then' propositions. We identified international evidence to test these propositions through searches of electronic databases and citation tracking, and supplemented our synthesis with evidence from similar forms of performance data. We synthesized this evidence through comparing the mechanisms and impact of patient-reported outcome measures and other performance data on quality improvement in different contexts. Results Three programme theories were identified: supporting patient choice, improving accountability and enabling providers to compare their performance with others. Relevant contextual factors were extent of public disclosure, use of financial incentives, perceived credibility of the data and the practicality of the results. Available evidence suggests that patients or their agents rarely use any published performance data when selecting a provider. The perceived motivation behind public reporting is an important determinant of how providers respond. When clinicians perceived that performance indicators were not credible but were incentivized to collect them, gaming or manipulation of data occurred. Outcome data do not provide information on the cause of poor care: providers needed to integrate and interpret patient-reported outcome measures and other outcome data in the context of other data. Lack of timeliness of performance data constrains their impact. Conclusions Although there is only limited research evidence to support some widely held theories of how aggregated patient-reported outcome measures data stimulate quality improvement, several lessons emerge from interventions sharing the same programme theories to help guide the increasing use of these measures.

Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care

Background

The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy.

Objectives

To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care.

Design

Two separate but related realist syntheses: (1) feedback of aggregate PROMs and performance data to improve patient care, and (2) feedback of individual PROMs data to improve patient care.

Interventions

Aggregate – feedback and public reporting of PROMs, patient experience data and performance data to hospital providers and primary care organisations. Individual – feedback of PROMs in oncology, palliative care and the care of people with mental health problems in primary and secondary care settings.

Main outcome measures

Aggregate – providers’ responses, attitudes and experiences of using PROMs and performance data to improve patient care. Individual – providers’ and patients’ experiences of using PROMs data to raise issues with clinicians, change clinicians’ communication practices, change patient management and improve patient well-being.

Data sources

Searches of electronic databases and forwards and backwards citation tracking.

Review methods

Realist synthesis to identify, test and refine programme theories about when, how and why PROMs feedback leads to improvements in patient care.

Results

Providers were more likely to take steps to improve patient care in response to the feedback and public reporting of aggregate PROMs and performance data if they perceived that these data were credible, were aimed at improving patient care, and were timely and provided a clear indication of the source of the problem. However, implementing substantial and sustainable improvement to patient care required system-wide approaches. In the care of individual patients, PROMs function more as a tool to support patients in raising issues with clinicians than they do in substantially changing clinicians’ communication practices with patients. Patients valued both standardised and individualised PROMs as a tool to raise issues, but thought is required as to which patients may benefit and which may not. In settings such as palliative care and psychotherapy, clinicians viewed individualised PROMs as useful to build rapport and support the therapeutic process. PROMs feedback did not substantially shift clinicians’ communication practices or focus discussion on psychosocial issues; this required a shift in clinicians’ perceptions of their remit.

Strengths and limitations

There was a paucity of research examining the feedback of aggregate PROMs data to providers, and we drew on evidence from interventions with similar programme theories (other forms of performance data) to test our theories.

Conclusions

PROMs data act as ‘tin openers’ rather than ‘dials’. Providers need more support and guidance on how to collect their own internal data, how to rule out alternative explanations for their outlier status and how to explore the possible causes of their outlier status. There is also tension between PROMs as a QI strategy versus their use in the care of individual patients; PROMs that clinicians find useful in assessing patients, such as individualised measures, are not useful as indicators of service quality.

Future work

Future research should (1) explore how differently performing providers have responded to aggregate PROMs feedback, and how organisations have collected PROMs data both for individual patient care and to improve service quality; and (2) explore whether or not and how incorporating PROMs into patients’ electronic records allows multiple different clinicians to receive PROMs feedback, discuss it with patients and act on the data to improve patient care.

Study registration

This study is registered as PROSPERO CRD42013005938.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Understanding and using quality information for quality improvement: The effect of information presentation

Objective

To examine how information presentation affects the understanding and use of information for quality improvement.

Design

An experimental design, testing 22 formats, and showing information on patient safety culture. Formats differed in visualization, outcomes and benchmark information.

Intervention(s)

Respondents viewed three randomly selected presentation formats in an online survey, completing several tasks per format.

Setting

The hospital sector in the Netherlands.

Participants

A volunteer sample of healthcare professionals, mainly nurses, working in hospitals. Main Outcome Measure(s): The degree to which information is understandable and usable (accurate choice for quality improvement, sense of urgency to change and appraisal of one's own performance).

Results

About 115 healthcare professionals participated (response rate 25%), resulting in 345 reviews.

Understandability

Information in tables (P = 0.007) and bar charts (P < 0.0001) was better understood than radars. Presenting outcomes on a 5-point scale (P < 0.001) or as '% positive responders' (P < 0.001) was better understood than '% negative responders'. Formats without benchmarks were better understood than formats with benchmarks. Use: Bar charts resulted in more accurate choices than tables (P = 0.003) and radars (P < 0.001). Outcomes on a 5-point scale resulted in more accurate choices than '% negative responders' (P = 0.007). Presenting '% positive responders' resulted in a higher sense of urgency to change than outcomes on a 5-point scale (P = 0.002). Benchmark information had inconsistent effects on the appraisal of one's own performances.

Conclusions

Information presentation affects healthcare professionals' understanding and use of quality information. Our findings supplement the further understanding on how quality information can be best communicated to healthcare professionals for realizing quality improvements.

The Nottingham Expectation and Complication score following Surgery (NECS): an universal scale for surgical outcome audit and peer comparison

BACKGROUND

Consultant Outcomes Publication (COP) is an NHS England initiative for promoting improvements in quality of care. However, at present outcomes are commonly expressed as mortality rates which do not necessarily reflect the performance of surgeons. We developed the Nottingham Expectation and Complication score following Surgery (NECS) to determine the success of surgical treatment from both the clinical perspective and the practical expectations agreed between surgeons and patients during the consent process.

METHOD

This was a pilot study to trial the use of the NECS score. It is a simple expression of overall outcome comprising three clinical domains: S - surgical outcome, T - surgical/technical complications and M - medical complications recorded by the treating clinician, and practical outcome determined by a joint clinical/patient assessment. 107 elective neurosurgical patients were included in this prospective study. 95 completed questionnaires were included.

RESULTS

75% patients achieved the best possible treatment score (S3T3M4). Of the 25% of patients who did not achieve this ideal outcome, the most common cause was either medical deterioration 18%, or technical complications of surgery discussed during the consent process 17%, or both. Surgeons rated their outcomes as expectations exceeded in 2% of cases, met in 92%, partially met in 5% and failed in 1%. Patients rated their outcomes as expectations exceeded in 37%, met in 37%, partially met in 18%, and 5% reported that their expectations were not met or they were worse than before the operation. Bivariate correlation analysis (Pearson's r coefficient) between overall 'expectation score' of patients and surgeons showed moderate correlation with r = .25 (p = .014).

CONCLUSION

NECS score can be used as an indicator to assess technical performance and patient satisfaction. It provides a more balanced quality indicator of the surgical service delivery than COP. It also offers additional advantages for auditing/planning improving care and may serve as an appraisal/revalidation tool.

Healthcare professionals' views on feedback of a patient safety culture assessment

BACKGROUND

By assessing patient safety culture, healthcare providers can identify areas for improvement in patient safety culture. To achieve this, these assessment outcomes have to be relevant and presented clearly. The aim of our study was to explore healthcare professionals' views on the feedback of a patient safety culture assessment.

METHODS

Twenty four hospitals participated in a patient safety culture assessment in 2012. Hospital departments received feedback in a report and on a website. In a survey, we evaluated healthcare professionals' views on this feedback and the effect of additional information about patient safety culture improvement strategies on the appraisal of the feedback. 20 hospitals participated in part I (evaluation of the report), 13 hospitals participated in part II (evaluation of the website).

RESULTS

Healthcare professionals (e.g. members of staff and department heads/managers) rated the feedback in the report and on the website positively (average mean on different aspects = 7.2 on a scale from 1 to 10). Interpreting results was sometimes difficult, and information was sometimes lacking, like specific recommendations and improvement strategies. The provision of additional general information on patient safety culture improvement strategies resulted only in a higher appraisal of the attractiveness (lay-out) of the report and the understandability of the feedback report. The majority (84 %) of the healthcare professionals agreed or partly agreed that the feedback on patient safety culture stimulated actions to improve patient safety culture. However, a quarter also stated that although the feedback report provided insight into the patient safety culture, they did not know how to improve patient safety culture in their hospital.

CONCLUSIONS

Healthcare professionals seem to be positive about the feedback on patient safety culture and its effect on stimulating patient safety culture improvement. To optimally tune feedback on patient safety culture towards healthcare professionals, the following might help: 1) pay attention to the understandability of outcomes for its intended users; and 2) create feedback that is tailored towards specific hospital departments.

Variation in Patient-reported Outcomes Across Hospitals Following Surgery

BACKGROUND

Although there is growing interest in applying patient-reported outcomes (PROs) toward surgical quality, the extent to which PROs vary across hospitals following surgical procedures is unknown.

OBJECTIVES

We examined variation in PROs, specifically health-related quality of life (HRQOL), across hospitals performing bariatric surgery.

RESEARCH DESIGN

A retrospective cohort study.

SUBJECTS

The Michigan Bariatric Surgery Collaborative is a statewide consortium of 39 hospitals performing laparoscopic gastric bypass, gastric banding, or sleeve gastrectomy (n=11,420 patients between 2008 and 2012).

MEASURES

We examined generic and disease-specific HRQOL measured by the Health and Activities Limitations Index (HALex) and Bariatric Quality of Life index (BQL) preoperatively and at 1 year. We measured the variation in postoperative HRQOL across hospitals, and the effect of risk and reliability adjustment on hospital ranking.

RESULTS

In this cohort, HRQOL varied by 56% (HALex) and 37% (BQL) across hospitals. Patient factors accounted for 58% (HALex) to 71% (BQL) of the variation in HRQOL across hospitals. After risk and reliability adjustment, HRQOL varied by 18% (by HALex) and 14.5% (by BQL) across hospitals, and the proportion of patients who experienced a large improvement in HRQOL by HALex ranged from 33% to 69% and 67% to 92% by BQL. After adjusting for patient factors and reliability, these differences diminished to 55%-64% (HALex) and 79%-84% (BQL).

CONCLUSIONS

Patient factors explain a large proportion of hospital-level variation in PROs following bariatric surgery, underscoring the importance of risk adjustment. However, some variation in PROs across hospitals remains unexplained, suggesting PROs may represent a viable indicator of hospital performance.

Integrating the use of patient-reported outcomes for both clinical practice and performance measurement: views of experts from 3 countries

Policy Points: The patient-reported outcome (PRO) is a standardized method for measuring patients' views of their health status. Our international study showed that experts in clinical practice and performance measurement supported the integrated collection of PRO data for use in both clinical care and performance measurement. The measurement of PROs to support patient-provider decisions and the use of PRO performance measures to evaluate health care providers have developed both separately and in parallel. The use of PROs would benefit from a shared vision by health care providers, purchasers of care, and patients regarding the aims and purposes of the various applications; and the establishment of trust among stakeholders concerning the prudent use of PRO performance measures.

CONTEXT

Patient-reported outcomes (PROs) can play an important role in patient-centered health care by focusing on the patient's health goals guiding therapeutic decisions. When aggregated, PROs also can be used for other purposes, including comparative effectiveness research, practice improvement, assessment of the performance of clinicians and organizations, and as a metric for value-based payments. The feasibility of integrating the use of PROs for these various purposes on a wide scale has not yet been demonstrated. Our study was conducted to inform policymakers of prudent next steps for implementing PROs in clinical practice and performance measurement programs in order to maximize their impact on the quality of health care.

METHODS

We conducted a qualitative study, interviewing 58 experts and leaders from 37 organizations (response rate: 88%) in the United States, England, and the Netherlands. Respondents included clinical practitioners (n = 30), measure developers (n = 11), and leaders of performance measurement programs (n = 17). We used a qualitative content analysis to assess current strategies for applying PROs in clinical practice and performance measurement and to identify barriers to and facilitators of further implementation.

FINDINGS

The use of PROs in clinical practice and for performance measurement has developed both separately and in parallel. Experts across the stakeholder spectrum support the collection of PRO data in an integrated manner that would enable using the data for these distinct purposes. We identified 2 main concerns about the feasibility for integrated use of PRO data: the complexity of establishing routine data collection and the tension among stakeholders when using PRO data for different purposes. These contrasting stakeholder views suggested varying interests among clinicians, measure developers, and purchasers of care.

CONCLUSIONS

Data collection approaches that support the use of PROs in health care are underdeveloped, need better integration with clinical care, and must be tailored to the characteristics of the health care system. Enabling the sustainable use of PROs will require a shared vision of clinical professionals, purchasers, and patients, with a prudent selection of the steps in implementing PROs that will maximize their impact on the quality of health care.

Health services research: evolution and applications

Health services research (HSR) is broadly focused on characterizing and improving the access, quality, delivery, and cost of health care. HSR is a multidisciplinary field, engaging experts in clinical medicine and surgery, policy, economics, implementation science, statistics, psychology, and education to improve the care of patients across all specialties. This article summarizes the evolution and distinctive attributes of HSR and present several real-world applications.