Impact of National Institutes of Health Gastrointestinal PROMIS Measures in Clinical Practice: Results of a Multicenter Controlled Trial

OBJECTIVES

The National Institutes of Health (NIH) created the Patient Reported Outcomes Measurement Information System (PROMIS) to allow efficient, online measurement of patient-reported outcomes (PROs), but it remains untested whether PROMIS improves outcomes. Here, we aimed to compare the impact of gastrointestinal (GI) PROMIS measures vs. usual care on patient outcomes.

METHODS

We performed a pragmatic clinical trial with an off-on study design alternating weekly between intervention (GI PROMIS) and control arms at one Veterans Affairs and three university-affiliated specialty clinics. Adults with GI symptoms were eligible. Intervention patients completed GI PROMIS symptom questionnaires on an e-portal 1 week before their visit; PROs were available for review by patients and their providers before and during the clinic visit. Usual care patients were managed according to customary practices. Our primary outcome was patient satisfaction as determined by the Consumer Assessment of Healthcare Providers and Systems questionnaire. Secondary outcomes included provider interpersonal skills (Doctors' Interpersonal Skills Questionnaire (DISQ)) and shared decision-making (9-item Shared Decision Making Questionnaire (SDM-Q-9)).

RESULTS

There were 217 and 154 patients in the GI PROMIS and control arms, respectively. Patient satisfaction was similar between groups (P>0.05). Intervention patients had similar assessments of their providers' interpersonal skills (DISQ 89.4±11.7 vs. 89.8±16.0, P=0.79) and shared decision-making (SDM-Q-9 79.3±12.4 vs. 79.0±22.0, P=0.85) vs.

CONCLUSIONS

This is the first controlled trial examining the impact of NIH PROMIS in clinical practice. One-time use of GI PROMIS did not improve patient satisfaction or assessment of provider interpersonal skills and shared decision-making. Future studies examining how to optimize PROs in clinical practice are encouraged before widespread adoption.

Computer versus physician identification of gastrointestinal alarm features

OBJECTIVE

It is important for clinicians to inquire about "alarm features" as it may identify those at risk for organic disease and who require additional diagnostic workup. We developed a computer algorithm called Automated Evaluation of Gastrointestinal Symptoms (AEGIS) that systematically collects patient gastrointestinal (GI) symptoms and alarm features, and then "translates" the information into a history of present illness (HPI). Our study's objective was to compare the number of alarms documented by physicians during usual care vs. that collected by AEGIS.

METHODS

We performed a cross-sectional study with a paired sample design among patients visiting adult GI clinics. Participants first received usual care by their physicians and then completed AEGIS. Each individual thus contributed both a physician-documented and computer-generated HPI. Blinded physician reviewers enumerated the positive alarm features (hematochezia, melena, hematemesis, unintentional weight loss, decreased appetite, and fevers) mentioned in each HPI. We compared the number of documented alarms within patient using the Wilcoxon signed-rank test.

RESULTS

Seventy-five patients had both physician and AEGIS HPIs. AEGIS identified more patients with positive alarm features compared to physicians (53% vs. 27%; p<.001). AEGIS also documented more positive alarms (median 1, interquartile range [IQR] 0-2) vs. physicians (median 0, IQR 0-1; p<.001). Moreover, clinicians documented only 30% of the positive alarms self-reported by patients through AEGIS.

CONCLUSIONS

Physicians documented less than one-third of red flags reported by patients through a computer algorithm. These data indicate that physicians may under report alarm features and that computerized "checklists" could complement standard HPIs to bolster clinical care.

Computer-generated vs. physician-documented history of present illness (HPI): results of a blinded comparison

OBJECTIVES

Healthcare delivery now mandates shorter visits with higher documentation requirements, undermining the patient-provider interaction. To improve clinic visit efficiency, we developed a patient-provider portal that systematically collects patient symptoms using a computer algorithm called Automated Evaluation of Gastrointestinal Symptoms (AEGIS). AEGIS also automatically "translates" the patient report into a full narrative history of present illness (HPI). We aimed to compare the quality of computer-generated vs. physician-documented HPIs.

METHODS

We performed a cross-sectional study with a paired sample design among individuals visiting outpatient adult gastrointestinal (GI) clinics for evaluation of active GI symptoms. Participants first underwent usual care and then subsequently completed AEGIS. Each individual thereby had both a physician-documented and a computer-generated HPI. Forty-eight blinded physicians assessed HPI quality across six domains using 5-point scales: (i) overall impression, (ii) thoroughness, (iii) usefulness, (iv) organization, (v) succinctness, and (vi) comprehensibility. We compared HPI scores within patient using a repeated measures model.

RESULTS

Seventy-five patients had both computer-generated and physician-documented HPIs. The mean overall impression score for computer-generated HPIs was higher than physician HPIs (3.68 vs. 2.80; P<0.001), even after adjusting for physician and visit type, location, mode of transcription, and demographics. Computer-generated HPIs were also judged more complete (3.70 vs. 2.73; P<0.001), more useful (3.82 vs. 3.04; P<0.001), better organized (3.66 vs. 2.80; P<0.001), more succinct (3.55 vs. 3.17; P<0.001), and more comprehensible (3.66 vs. 2.97; P<0.001).

CONCLUSIONS

Computer-generated HPIs were of higher overall quality, better organized, and more succinct, comprehensible, complete, and useful compared with HPIs written by physicians during usual care in GI clinics.

Dialysis practices that distinguish top- versus bottom-performing facilities by hemoglobin outcomes

BACKGROUND

Because there is wide variation in outcomes across dialysis facilities, it is possible that top-performing units use practices not shared by others. The Identifying Best Practices in Dialysis (IBPiD) Study seeks to identify practices that distinguish top- from bottom-performing facilities by key outcomes, including achievement of recommended hemoglobin targets.

STUDY DESIGN

Observational study with cross-sectional study ascertainment of predictors and outcomes.

PREDICTORS

Facility dialysis practices ascertained using practice surveys of dialysis staff who indicated their level of agreement that each practice occurs in their facility (1-6 on a Likert scale).

SETTING & PARTICIPANTS

423 personnel in 90 dialysis facilities from 1 for-profit and 2 not-for-profit dialysis organizations.

OUTCOMES

Percentage of patients per month per facility with hemoglobin levels of 11-12 g/dL. We divided facilities by median into top- versus bottom-performing groups and compared mean scores for each practice using t tests. We report practices that were statistically significant and achieved at least a medium effect size (ES) >or=0.4.

RESULTS

17 of 155 tested predictors were significant. Achievement of hemoglobin level targets was related most strongly to the use of chairside computers (ES, 0.8 [95% CI, 0.4-1.4]), extent/quality of educational videos (ES, 0.6 [95% CI, 0.2-1.1]), frequency of calling per diem staff if short staffed (ES, 0.6 [95% CI, 0.21-1.1]), policy that nurses pass written competency examinations before hire (ES, 0.6 [95% CI, 0.2-1.0]), and technician cannulation mastery (ES, 0.6 [95% CI, 0.2-1.1]).

LIMITATIONS

This is a cross-sectional study that can address only associations, not causations. Future research should measure the longitudinal predictive value of these practices.

CONCLUSIONS

High-performing facilities report more effective education programs, better staff management, higher staff competency, and higher use of chairside computers, a potential marker of information technology proficiency. This suggests that hemoglobin level management is enhanced by processes reflecting a coordinated multidisciplinary environment.

Is there "cherry picking" in the ESRD Program? Perceptions from a Dialysis Provider Survey

BACKGROUND AND OBJECTIVES

Changes in ESRD reimbursement policy, including proposed bundled payment, have raised concern that dialysis facilities may use "cherry picking" practices to attract a healthier, better insured, or more adherent patient population.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

As part of a national survey to measure beliefs about drivers of quality in dialysis, respondents were asked about their perceptions of cherry picking, including the frequency and effect of various cherry picking strategies on dialysis outcomes. We surveyed a random sample of 250 nurse members of the American Nephrology Nurses Association, 250 nephrologist members of the American Medical Association, 50 key opinion leaders, and 2000 physician members of the Renal Physicians Association. We tested hypothesized predictors of perception, including provider group, region, age, experience, and the main practice facility features.

RESULTS

Three-quarters of respondents reported that cherry picking occurred "sometimes" or "frequently." There were no differences in perceptions by provider or facility characteristics, insurance status, or health status. In multivariable regression, perceived cherry picking was 2.8- and 3.5-fold higher in the northeast and Midwest, respectively, versus the west. Among various cherry picking strategies, having a "low threshold to 'fire' chronic no-shows/late arrivers," and having a "low threshold to 'fire' for noncompliance with diet and meds" had the largest perceived association with outcomes.

CONCLUSIONS

Under current reimbursement practices, dialysis caregivers perceive that cherry picking is common and important. An improved understanding of cherry picking practices, if evident, may help to protect vulnerable patients if reimbursement practices were to change.

Identifying best practices in dialysis care: results of cognitive interviews and a national survey of dialysis providers

BACKGROUND AND OBJECTIVES

Because there is wide variation in case-mix adjusted outcomes across dialysis facilities, it is possible that top-performing facilities use practices not shared by others. We sought to catalogue "best practices" that may account for interfacility variations in outcomes.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

This multidisciplinary study identified candidate best practices in dialysis through a staged process, including systematic review, cognitive interviews, and a national "virtual focus group" of dialysis providers. The resulting candidate practices were rank-ordered by perceived importance as determined by mean RAND Appropriateness Scores from a national survey of nephrologists, nurses, and opinion leaders.

RESULTS

A total of 155 candidate best practices were identified. Among these, respondents believed dialysis outcomes are most strongly related to 1) characteristics of multidisciplinary care conferences, 2) technician proficiency in protecting vascular access, 3) training of nurses to provide education in fluid management, vascular access, and nutrition, 4) use of random and blinded audits of staff performance, and 5) communication and teamwork among staff. In contrast, there was wide disagreement about the importance of facility-based health maintenance practices, optimal staffing ratios, frequency of dialysis-based physician visits, and optimal frequency of multidisciplinary care.

CONCLUSIONS

This study provides a "conceptual map" of candidate dialysis best practices and highlights areas of general agreement and disagreement. These findings can help the dialysis community think critically about what may define "best practice" and provide targets for future research in quality improvement.