Measuring patients' experiences with individual specialist physicians and their practices

The Reliability of Graduate Medical Education Quality of Care Clinical Performance Measures

BACKGROUND

Graduate medical education (GME) program leaders struggle to incorporate quality measures in the ambulatory care setting, leading to knowledge gaps on how to provide feedback to residents and programs. While nationally collected quality of care data are available, their reliability for individual resident learning and for GME program improvement is understudied.

OBJECTIVE

To examine the reliability of the Healthcare Effectiveness Data and Information Set (HEDIS) clinical performance measures in family medicine and internal medicine GME programs and to determine whether HEDIS measures can inform residents and their programs with their quality of care.

METHODS

From 2014 to 2017, we collected HEDIS measures from 566 residents in 8 family medicine and internal medicine programs under one sponsoring institution. Intraclass correlation was performed to establish patient sample sizes required for 0.70 and 0.80 reliability levels at the resident and program levels. Differences between the patient sample sizes required for reliable measurement and the actual patients cared for by residents were calculated.

RESULTS

The highest reliability levels for residents (0.88) and programs (0.98) were found for the most frequently available HEDIS measure, colorectal cancer screening. At the GME program level, 87.5% of HEDIS measures had sufficient sample sizes for reliable measurement at alpha 0.7 and 75.0% at alpha 0.8. Most resident level measurements were found to be less reliable.

CONCLUSIONS

GME programs may reliably evaluate HEDIS performance pooled at the program level, but less so at the resident level due to patient volume.

Impact of patient characteristics on the Canadian Patient Experiences Survey-Inpatient Care: survey analysis from an academic tertiary care centre

OBJECTIVE

To determine the role of patient demographics, care domains and self-perceived health status in the analysis and interpretation of results from the Canadian Patient Experience Survey-Inpatient Care.

DESIGN

Cross-sectional survey.

SETTING

Single large Canadian two campus tertiary care academic centre.

PARTICIPANTS

Random sampling of hospital patients postdischarge.

INTERVENTION AND MAIN OUTCOME MEASURES

Logistic regression models were developed to analyse topbox scoring on four questions of global care (rate experience, recommend hospital, rate hospital, overall helped). Means of each composite domain were correlated to the four overall scores at the patient level to determine Spearman's rank correlation coefficients which were plotted against the overall (hospital) domain score for the key driver analysis.

RESULTS

Topbox scoring was decreased with worse degrees of perceived physical and mental health in all four global questions (p<0.05). Female gender and higher levels of education were associated with worse scoring on rate experience, recommend hospital and rate hospital (p<0.001). Whereas there was a significant difference between hospital departments in unadjusted measures, these differences were no longer evident after adjustment with patient covariates. Key driver analysis identified person-centred care, care transition and the domain related to emergency admission as areas of highest potential for improvement.

CONCLUSIONS

Global measures of overall care are influenced by patient-perceived physical and mental health. Caution should be exercised in using patient-satisfaction surveys to compare performance between different healthcare provision entities, as apparent differences could be explained by variation in patient mix rather than variation in performance.

Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data

OBJECTIVES

Recent public reporting initiatives in England highlight general practice variation in indicators of diagnostic activity related to cancer. We aimed to quantify the size and sources of variation and the reliability of practice-level estimates of such indicators, to better inform how this information is interpreted and used for quality improvement purposes.

DESIGN

Ecological cross-sectional study.

SETTING

English primary care.

PARTICIPANTS

All general practices in England with at least 1000 patients.

MAIN OUTCOME MEASURES

Sixteen diagnostic activity indicators from the Cancer Services Public Health Profiles.

RESULTS

Mixed-effects logistic and Poisson regression showed that substantial proportions of the observed variance in practice scores reflected chance, variably so for different indicators (between 7% and 85%). However, after accounting for the role of chance, there remained substantial variation between practices (typically up to twofold variation between the 75th and 25th centiles of practice scores, and up to fourfold variation between the 90th and 10th centiles). The age and sex profile of practice populations explained some of this variation, by different amounts across indicators. Generally, the reliability of diagnostic process indicators relating to broader populations of patients most of whom do not have cancer (eg, rate of endoscopic investigations, or urgent referrals for suspected cancer (also known as 'two week wait referrals')) was high (≥0.80) or very high (≥0.90). In contrast, the reliability of diagnostic outcome indicators relating to incident cancer cases (eg, per cent of all cancer cases detected after an emergency presentation) ranged from 0.24 to 0.54, which is well below recommended thresholds (≥0.70).

CONCLUSIONS

Use of indicators of diagnostic activity in individual general practices should principally focus on process indicators which have adequate or high reliability and not outcome indicators which are unreliable at practice level.

Reliability of Physician-Level Measures of Patient Experience in Primary Care

BACKGROUND

Patient experience measures are widely used to compare performance at the individual physician level.

OBJECTIVE

To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level.

DESIGN

Repeated cross-sectional design.

SETTING

Academic family medicine practice in California.

PARTICIPANTS

One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians).

MEASUREMENTS

Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status).

RESULTS

The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R² = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0-25.2%)].

CONCLUSIONS

Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.

Influences on patient satisfaction in healthcare centers: a semi-quantitative study over 5 years

BACKGROUND

Knowledge of ambulatory patients' satisfaction with clinic visits help improve communication and delivery of healthcare. The goal was to examine patient satisfaction in a primary care setting, identify how selected patient and physician setting and characteristics affected satisfaction, and determine if feedback provided to medical directors over time impacted patient satisfaction.

METHODS

A three-phase, semi-quantitative analysis was performed using anonymous, validated patient satisfaction surveys collected from 889 ambulatory outpatients in 6 healthcare centers over 5-years. Patients' responses to 21 questions were analyzed by principal components varimax rotated factor analysis. Three classifiable components emerged: Satisfaction with Physician, Availability/Convenience, and Orderly/Time. To study the effects of several independent variables (location of clinics, patients' and physicians' age, education level and duration at the clinic), data were subjected to multivariate analysis of variance (MANOVA)..

RESULTS

Changes in the healthcare centers over time were not significantly related to patient satisfaction. However, location of the center did affect satisfaction. Urban patients were more satisfied with their physicians than rural, and inner city patients were less satisfied than urban or rural on Availability/Convenience and less satisfied than urban patients on Orderly/Time. How long a patient attended a center most affected satisfaction, with patients attending >10 years more satisfied in all three components than those attending <1-5 years. Level of education affected patients' satisfaction only in the component Orderly/Time; patients without a high school education were significantly less satisfied than those with more. Patients in their 40's were significantly less satisfied in Availability/Convenience than those >60 years old. Patients were significantly more satisfied with their 30-40 year-old physicians compared with those over 60. On Orderly/Time, patients were more satisfied with physicians who were in their 50's than physicians >60.

CONCLUSIONS

Improvement in patient satisfaction includes a need for immediate, specific feedback. Although Medical Directors received feedback yearly, we found no significant changes in patient satisfaction over time. Our results suggest that, to increase satisfaction, patients with lower education, those who are sicker, and those who are new to the center likely would benefit from additional high quality interactions with their physicians.

Does Race/Ethnicity or Socioeconomic Status Influence Patient Satisfaction in Pediatric Surgical Care?

OBJECTIVE

To evaluate patient satisfaction in outpatient pediatric surgical care and assess differences in scores by race/ethnicity and socioeconomic status (SES).

STUDY DESIGN

Observational, cross-sectional analysis.

SETTING

Outpatient pediatric surgical specialty clinics at a tertiary academic center.

SUBJECT AND METHODS

Families of patients received a patient satisfaction survey following their initial care visit in 2012. Mean scores were calculated and compared by child race/ethnicity and insurance type, where insurance with medical assistance (MA) served as a proxy for low SES. Kruskal-Wallis tests were used to compare scores between groups. Surveys were dichotomized to low and high scorers, and multivariate logistic regression was used to calculate the likelihood of high satisfaction.

RESULTS

Of 527 surveys completed, 132 (25%) were for children with MA and 143 (27%) were for racial/ethnic minority children. The overall satisfaction score for all specialties was 84.8, which did not significantly differ by SES (P = .98) or minority status (P = .52). The survey item with the highest score in both SES groups was "degree to which provider talked with you using words you could understand" (overall mean 91.94, P = .23). Multivariate analysis showed that patient age, sex, race/ethnicity, insurance type, neighborhood SES, neighborhood diversity, or surgical department did not significantly influence satisfaction.

CONCLUSION

This is the first study to evaluate the relationship between SES and race/ethnicity with patient satisfaction in outpatient pediatric surgical specialty care. In this analysis, no disparities were identified in the patient experience by individual- or community-level factors. Although the survey methodologies may be limited, these findings suggest that provision of care in pediatric surgical specialties can be simultaneously equitable, culturally competent, and family centered.

Using segmented regression analysis of interrupted time series data to assess colonoscopy quality outcomes of a web-enhanced implementation toolkit to support evidence-based practices for bowel preparation: a study protocol

BACKGROUND

While there is convincing evidence on interventions to improve bowel preparation for patients, the evidence on how to implement these evidence-based practices (EBPs) in outpatient colonoscopy settings is less certain. The Strategies to Improve Colonoscopy (STIC) study compares the effect of two implementation strategies, physician education alone versus physician education plus an implementation toolkit for staff, on adoption of three EBPs (split-dosing of bowel preparation, low-literacy education, teach-back) to improve pre-procedure and intra-procedure quality measures. The implementation toolkit contains a staff education module, website containing tools to support staff in delivering EBPs, tailored patient education materials, and brief consultation with staff to determine how the EBPs can be integrated into the existing workflow. Given adaptations to the implementation plan and intentional flexibility in the delivery of the EBPs, we utilize a pragmatic study to balance external validity with demonstrating effectiveness of the implementation strategies.

METHODS/DESIGN

Participants will include all outpatient colonoscopy physicians, staff, and patients from a convenience sample of six endoscopy settings. Aim #1 will explore the relative effect of two strategies to implement patient-level EBPs on adoption and clinical quality outcomes. We will assess the change in level and trends of clinical quality outcomes (i.e., adequacy of bowel preparation, adenoma detection) using segmented regression analysis of interrupted time series data with two groups (intervention and delayed start). Aim #2 will examine the influence of organizational readiness to change on EBP implementation. We use a PRECIS diagram to reflect the extent to which each indicator of the study was pragmatic versus explanatory, revealing a largely pragmatic study.

DISCUSSION

Implementation challenges have already motivated several adaptations to the original plan, reflecting the nature of implementation in real-world healthcare settings. The pragmatic study responds to the evolving needs of its healthcare partners and allows for flexibility in intervention delivery, thereby informing clinical decision-making in real-world settings. The current study will provide information about what works (intervention effectiveness), for whom it works (influence of Medicaid versus other insurance), in which contexts it works (setting characteristics that influence implementation), and how it works best (comparison of implementation strategies).

Specialties differ in which aspects of doctor communication predict overall physician ratings

BACKGROUND

Effective doctor communication is critical to positive doctor-patient relationships and predicts better health outcomes. Doctor communication is the strongest predictor of patient ratings of doctors, but the most important aspects of communication may vary by specialty.

OBJECTIVE

To determine the importance of five aspects of doctor communication to overall physician ratings by specialty.

DESIGN

For each of 28 specialties, we calculated partial correlations of five communication items with a 0-10 overall physician rating, controlling for patient demographics.

PATIENTS

Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS®) 12-month Survey data collected 2005-2009 from 58,251 adults at a 534-physician medical group.

MAIN MEASURES

CG-CAHPS includes a 0 ("Worst physician possible") to 10 ("Best physician possible") overall physician rating. Five doctor communication items assess how often the physician: explains things; listens carefully; gives easy-to-understand instructions; shows respect; and spends enough time.

KEY RESULTS

Physician showing respect was the most important aspect of communication for 23/28 specialties, with a mean partial correlation (0.27, ranging from 0.07 to 0.44 across specialties) that accounted for more than four times as much variance in the overall physician rating as any other communication item. Three of five communication items varied significantly across specialties in their associations with the overall rating (p < 0.05).

CONCLUSIONS

All patients valued respectful treatment; the importance of other aspects of communication varied significantly by specialty. Quality improvement efforts by all specialties should emphasize physicians showing respect to patients, and each specialty should also target other aspects of communication that matter most to their patients. The results have implications for improving provider quality improvement and incentive programs and the reporting of CAHPS data to patients. Specialists make important contributions to coordinated patient care, and thus customized approaches to measurement, reporting, and quality improvement efforts are important.

A first approach to differences in continuity of care perceived by immigrants and natives in the Catalan public healthcare system

Objective: To compare immigrants’ and natives’ perceptions of relational, managerial and informational continuity of care and to explore the influence of the length of stay on immigrants’ perceptions of continuity. Methods: Cross-sectional study based on a survey of a random sample of 1,500 patients, of which 22% (331) were immigrants. The study area was made up by three healthcare areas of the Catalan healthcare system. To collect data, the CCAENA questionnaire was applied. Multivariate logistic regression models were conducted. Results: Like natives, immigrants perceive high levels of managerial continuity (88.5%) and relational continuity with primary and secondary care physicians (86.7 and 81.8%), and lower levels of informational continuity (59.1%). There were no statistically significant differences in managerial and informational continuity between immigrants and natives. However, immigrants perceive a worse relational continuity with primary care physicians in terms of trust, communication and clinical responsibility. Conversely, immigrants perceive higher relational continuity with secondary care physicians in terms of effective communication and clinical responsibility. Discussion: Similar managerial and informational continuity perceptions seem to point towards a similar treatment of patients, regardless of their immigrant status. However, differences in relational continuity highlight the need for improvements in professionals’ skills in treating immigrants’ patients.

Factors associated to experienced continuity of care between primary and outpatient secondary care in the Catalan public healthcare system

OBJECTIVE

To analyze patient's reported elements of relational, informational and managerial (dis)continuity between primary and outpatient secondary care and to identify associated factors.

METHODS

Cross-sectional study by means of a survey of a random sample of 1500 patients attended in primary and secondary care for the same condition. The study settings consisted of three health areas of the Catalan health system. Data were collected in 2010 using the CCAENA questionnaire, which identifies patients' experiences of continuity of care. Descriptive analyses and multivariable logistic regression models were carried out.

RESULTS

Elements of continuity of care were experienced by most patients. However, elements of discontinuity were also identified: 20% and 15% were seen by more than one primary or secondary care physician, respectively. Their secondary care physician or both professionals were identified as responsible for their care by 40% and 45% of users, respectively. Approximately 20% reported a lack of information transfer. Finally, 72% of secondary care consultations were due to primary care referral, whilst only 36% reported a referral back to primary care. Associated factors were healthcare setting, age, sex, perceived health status and disease duration.

CONCLUSION

Users generally reported continuity of care, although elements of discontinuity were also identified, which can be partially explained by the healthcare setting and some individual factors. Elements of discontinuity should be addressed to better adapt care to patients' needs.

How can health care organizations be reliably compared?: Lessons from a national survey of patient experience

BACKGROUND

Patient experience is increasingly used to assess organizational performance, for example in public reporting or pay-for-performance schemes. Conventional approaches using 95% confidence intervals are commonly used to determine required survey samples or to report performance but these may result in unreliable organizational comparisons.

METHODS

We analyzed data from 2.2 million patients who responded to the English 2009 General Practice Patient Survey, which included 45 patient experience questions nested within 6 different care domains (access, continuity of care, communication, anticipatory care planning, out-of-hours care, and overall care satisfaction). For each question, unadjusted and case-mix adjusted (for age, sex, and ethnicity) organization-level reliability, and intraclass correlation coefficients were calculated.

RESULTS

Mean responses per organization ranged from 23 to 256 for questions evaluating primary care practices, and from 1454 to 2758 for questions evaluating out-of-hours care organizations. Adjusted and unadjusted reliability values were similar. Twenty-six questions had excellent reliability (≥0.90). Seven nurse communication questions had very good reliability (≥0.85), but 3 anticipatory care planning questions had lower reliability (<0.70). Reliability was typically <0.70 for questions with <100 mean responses per practice, usually indicating questions which only a subset of patients were eligible to answer. Nine questions had both excellent reliability and high intraclass correlation coefficients (≥0.10) indicating both reliable measurement and substantial performance variability.

CONCLUSIONS

High reliability is a necessary property of indicators used to compare health care organizations. Using the English General Practice Patient Survey as a case study, we show how reliability and intraclass correlation coefficients can be used to select measures to support robust organizational comparisons, and to design surveys that will both provide high-quality measurement and optimize survey costs.

The General Practitioner Experiences Questionnaire (GPEQ): validity and reliability following the inclusion of new accessibility items

BACKGROUND

The General Practitioner Experiences Questionnaire (GPEQ) measures community mental health centres from the perspective of GP but lacked accessibility scales and documentation of centre level reliability.

OBJECTIVES

To assess the psychometric properties of the GPEQ following the inclusion of four new accessibility items and estimate centre level reliability for all scales and items.

METHODS

The design of the study is cross-sectional national survey. The setting of the study is postal survey of GPs in Norway evaluating 80 community mental health centres in the four health regions in Norway during autumn of 2008. Three thousand nine hundred and forty-two GPs were sent a postal questionnaire with the GPEQ and were asked to assess their community mental health centre responsible for general adult psychiatric services.

RESULTS

Two thousand two hundred and nine (56.0%) GPs returned a completed questionnaire. Psychometric testing including factor analysis and internal consistency reliability identified seven scales with satisfactory reliability and validity: accessibility (two items, new scale), competence (four items), discharge letter (three items), emergency situations (two items), guidance (three items), referrals (three items, new scale) and workforce situation (four items). All scales met the criterion of 0.7 for Cronbach's alpha. The centre level reliability was >0.8 for all scales, while all items met the criterion of 0.7.

CONCLUSIONS

The inclusion of new accessibility items resulted in two new scales in addition to the original five scales in the GPEQ, serving to improve the content validity of the instrument. Centre level reliability was high for all scales and items, giving further support to use GPEQ scores as quality indicators.

A randomized trial comparing mail versus in-office distribution of the CAHPS Clinician and Group Survey

OBJECTIVE

To assess the effect of survey distribution protocol (mail versus handout) on data quality and measurement of patient care experiences.

DATA SOURCES/STUDY SETTING

Multisite randomized trial of survey distribution protocols. Analytic sample included 2,477 patients of 15 clinicians at three practice sites in New York State.

DATA COLLECTION/EXTRACTION METHODS

Mail and handout distribution modes were alternated weekly at each site for 6 weeks.

PRINCIPAL FINDINGS

Handout protocols yielded an incomplete distribution rate (74 percent) and lower overall response rates (40 percent versus 58 percent) compared with mail. Handout distribution rates decreased over time and resulted in more favorable survey scores compared with mailed surveys. There were significant mode-physician interaction effects, indicating that data cannot simply be pooled and adjusted for mode.

CONCLUSIONS

In-office survey distribution has the potential to bias measurement and comparison of physicians and sites on patient care experiences. Incomplete distribution rates observed in-office, together with between-office differences in distribution rates and declining rates over time suggest staff may be burdened by the process and selective in their choice of patients. Further testing with a larger physician and site sample is important to definitively establish the potential role for in-office distribution in obtaining reliable, valid assessment of patient care experiences.

The Schwartz Center Rounds: evaluation of an interdisciplinary approach to enhancing patient-centered communication, teamwork, and provider support

PURPOSE

To assess the impact of Schwartz Center Rounds, an interdisciplinary forum where attendees discuss psychosocial and emotional aspects of patient care. The authors investigated changes in attendees' self-reported behaviors and beliefs about patient care, sense of teamwork, stress, and personal support.

METHOD

In 2006-2007, researchers conducted retrospective surveys of attendees at six sites offering Schwartz Center Rounds ("the Rounds") for > or =3 years and prospective surveys of attendees at 10 new Rounds sites that have held > or =7 Rounds.

RESULTS

Most of the retrospective survey respondents indicated that attending Rounds enhanced their likelihood of attending to psychosocial and emotional aspects of care and enhanced their beliefs about the importance of empathy. Respondents reported better teamwork, including heightened appreciation of the roles and contributions of colleagues. There were significant decreases in perceived stress (P < .001) and improvements in the ability to cope with the psychosocial demands of care (P < .05). In the prospective study, after control for presurvey differences, the more Rounds one attended, the greater the impact on postsurvey insights into psychosocial aspects of care and teamwork (both: P < .05). Respondents to both retrospective and prospective surveys described changes in institutional culture and greater focus on patient-centered care and institution-specific initiatives.

CONCLUSIONS

Schwartz Center Rounds may foster enhanced communication, teamwork, and provider support. The impact on measured outcomes increased with the number of Rounds attended. The Rounds represent an effective strategy for providing support to health care professionals and for enhancing relationships among them and with their patients.

The effect of performance-based financial incentives on improving patient care experiences: a statewide evaluation

BACKGROUND

Patient experience measures are central to many pay-for-performance (P4P) programs nationally, but the effect of performance-based financial incentives on improving patient care experiences has not been assessed.

METHODS

The study uses Clinician & Group CAHPS data from commercially insured adult patients (n = 124,021) who had visits with 1,444 primary care physicians from 25 California medical groups between 2003 and 2006. Medical directors were interviewed to assess the magnitude and nature of financial incentives directed at individual physicians and the patient experience improvement activities adopted by groups. Multilevel regression models were used to assess the relationship between performance change on patient care experience measures and medical group characteristics, financial incentives, and performance improvement activities.

RESULTS

Over the course of the study period, physicians improved performance on the physician-patient communication (0.62 point annual increase, p < 0.001), care coordination (0.48 point annual increase, p < 0.001), and office staff interaction (0.22 point annual increase, p = 0.02) measures. Physicians with lower baseline performance on patient experience measures experienced larger improvements (p < 0.001). Greater emphasis on clinical quality and patient experience criteria in individual physician incentive formulas was associated with larger improvements on the care coordination (p < 0.01) and office staff interaction (p < 0.01) measures. By contrast, greater emphasis on productivity and efficiency criteria was associated with declines in performance on the physician communication (p < 0.01) and office staff interaction (p < 0.001) composites.

CONCLUSIONS

In the context of statewide measurement, reporting, and performance-based financial incentives, patient care experiences significantly improved. In order to promote patient-centered care in pay for performance and public reporting programs, the mechanisms by which program features influence performance improvement should be clarified.

Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England

OBJECTIVE

To assess the robustness of patient responses to a new national survey of patient experience as a basis for providing financial incentives to doctors.

DESIGN

Analysis of the representativeness of the respondents to the GP Patient Survey compared with those who were sampled (5.5 million patients registered with 8273 general practices in England in January 2009) and with the general population. Analysis of non-response bias looked at the relation between practice response rates and scores on the survey. Analysis of the reliability of the survey estimated the proportion of the variance of practice scores attributable to true differences between practices.

RESULTS

The overall response rate was 38.2% (2.2 million responses), which is comparable to that in surveys using similar methodology in the UK. Men, young adults, and people living in deprived areas were under-represented among respondents. However, for questions related to pay for performance, there was no systematic association between response rates and questionnaire scores. Two questions which triggered payments to general practitioners were reliable measures of practice performance, with average practice-level reliability coefficients of 93.2% and 95.0%. Less than 3% and 0.5% of practices had fewer than the number of responses required to achieve conventional reliability levels of 90% and 70%. A change to the payment formula in 2009 resulted in an increase in the average impact of random variation in patient scores on payments to general practitioners compared with payments made in 2007 and 2008.

CONCLUSIONS

There is little evidence to support the concern of some general practitioners that low response rates and selective non-response bias have led to systematic unfairness in payments attached to questionnaire scores. The study raises issues relating to the validity and reliability of payments based on patient surveys and provides lessons for the UK and for other countries considering the use of patient experience as part of pay for performance schemes.

Attributing sources of variation in patients' experiences of ambulatory care

BACKGROUND

Public reporting and pay-for-performance programs increasingly rely on patient experience data to evaluate individual physicians and guide quality improvement efforts. The extent to which performance variation is attributable to physicians versus other system-level units, however, remains unclear.

METHODS

Using ambulatory care experience survey data from 61,839 patients of 1729 primary care physicians in California (response rate = 39.1%), this study assesses the proportion of explainable performance variation attributable to various organizational units in composite measures of physician-patient interaction, organizational features of care, and global assessments of care. For each measure, multilevel regression models that controlled for respondent characteristics and used random effects to account for the clustering of patients within physicians, physicians within care sites, care sites within medical groups, and medical groups within primary care service areas, estimated the proportion of explainable performance variation attributable to each system-level unit.

RESULTS

System-level factors explained between 27.9% to 47.7% of variation, with the highest proportion explained for the access to care composite and the lowest explained for the quality of chronic care composite. Physicians accounted for the largest proportion of explainable variance for all measures (range: 35.1%-49.0%). Care sites and primary care service areas explained substantial proportions of variance (>20% each) for the access to care and care coordination measures. Medical groups explained the largest proportions of variation (>20%) for global assessments of care.

CONCLUSIONS

Individual physicians and their care sites are the most important foci for patient experience improvement efforts. Because markets contribute substantially to performance variation on organizational features of care, future research should clarify the extent to which associated performance deficits are modifiable.

Expectations and experiences of eHealth in primary care: a qualitative practice-based investigation

OBJECTIVES

(1) To assess expectations and experiences of a new eHealth service by patients and staff in three primary care settings; (2) to ascertain attitudes to a range of future, primary care-oriented eHealth services.

DESIGN

Qualitative case study.

SETTING

Three UK general practices introducing an eHealth service for booking patient appointments.

PARTICIPANTS

Ninety patients purposively selected from users and non-users of the new service and 28 staff (clinicians, management and administrative staff).

RESULTS

Actual patient use of the service was lower than stated intention. Patients and staff felt that more active promotion of the service would have resulted in more use. Low usage did not result in a negative assessment of the service by most staff. Different patient groupings were identified with characteristics that may be used as predictors of eHealth service use and indicators of training needs. GPs and patients expressed opposing viewpoints on a range of future eHealth services.

CONCLUSIONS

Take-up of eHealth services may be lower than expected. To overcome patient barriers, factors that may narrow the intention-behaviour gap such as level of service promotion, GP endorsement, and usage by different patient groups, should be investigated. For clinician barriers, the eHealth evidence base needs strengthening, while for primary care practices, a learning process including staff training needs to be instituted. The differing views of patients and GPs about components of eHealth means that policymakers need to plan for a lengthy political process to obtain agreement on contentious issues if they are to achieve successful eHealth services.