Numeracy and Understanding of Quantitative Aspects of Predictive Models: A Pilot Study

Clinician Perspectives on Barriers and Enablers to Implementing an Inpatient Oncology Early Warning System: A Mixed-Methods Study

PURPOSE

To elicit end-user and stakeholder perceptions regarding design and implementation of an inpatient clinical deterioration early warning system (EWS) for oncology patients to better fit routine clinical practices and enhance clinical impact.

METHODS

In an explanatory-sequential mixed-methods study, we evaluated a stakeholder-informed oncology early warning system (OncEWS) using surveys and semistructured interviews. Stakeholders were physicians, advanced practice providers (APPs), and nurses. For qualitative data, we used grounded theory and thematic content analysis via the constant comparative method to identify determinants of OncEWS implementation.

RESULTS

Survey respondents generally agreed that an oncology-focused EWS could add value beyond clinical judgment, with nurses endorsing this notion significantly more strongly than other clinicians (nurse: median 5 on a 6-point scale [6 = strongly agree], interquartile range 4-5; doctors/advanced practice providers: 4 [4-5]; P = .005). However, some respondents would not trust an EWS to identify risk accurately (n = 36 [42%] somewhat or very concerned), while others were concerned that institutional culture would not embrace such an EWS (n = 17 [28%]).Interviews highlighted important aspects of the EWS and the local context that might facilitate implementation, including (1) a model tailored to the subtleties of oncology patients, (2) transparent model information, and (3) nursing-centric workflows. Interviewees raised the importance of sepsis as a common and high-risk deterioration syndrome.

CONCLUSION

Stakeholders prioritized maximizing the degree to which the OncEWS is understandable, informative, actionable, and workflow-complementary, and perceived these factors to be key for translation into clinical benefit.

Assessing Data Adequacy for High Blood Pressure Clinical Decision Support: A Quantitative Analysis

OBJECTIVE

This study examines guideline-based high blood pressure (HBP) and hypertension recommendations and evaluates the suitability and adequacy of the data and logic required for a Fast Healthcare Interoperable Resources (FHIR)-based, patient-facing clinical decision support (CDS) HBP application. HBP is a major predictor of adverse health events, including stroke, myocardial infarction, and kidney disease. Multiple guidelines recommend interventions to lower blood pressure, but implementation requires patient-centered approaches, including patient-facing CDS tools.

METHODS

We defined concept sets needed to measure adherence to 71 recommendations drawn from eight HBP guidelines. We measured data quality for these concepts for two cohorts (HBP screening and HBP diagnosed) from electronic health record (EHR) data, including four use cases (screening, nonpharmacologic interventions, pharmacologic interventions, and adverse events) for CDS.

RESULTS

We identified 102,443 people with diagnosed and 58,990 with undiagnosed HBP. We found that 21/35 (60%) of required concept sets were unused or inaccurate, with only 259 (25.3%) of 1,101 codes used. Use cases showed high inclusion (0.9-11.2%), low exclusion (0-0.1%), and missing patient-specific context (up to 65.6%), leading to data in 2/4 use cases being insufficient for accurate alerting.

DISCUSSION

Data quality from the EHR required to implement recommendations for HBP is highly inconsistent, reflecting a fragmented health care system and incomplete implementation of standard terminologies and workflows. Although imperfect, data were deemed adequate for two test use cases.

CONCLUSION

Current data quality allows for further development of patient-facing FHIR HBP tools, but extensive validation and testing is required to assure precision and avoid unintended consequences.

Preferences for Predictive Model Characteristics among People Living with Chronic Lung Disease: A Discrete Choice Experiment

Background. Patients may find clinical prediction models more useful if those models accounted for preferences for false-positive and false-negative predictive errors and for other model characteristics. Methods. We conducted a discrete choice experiment to compare preferences for characteristics of a hypothetical mortality prediction model among community-dwelling patients with chronic lung disease recruited from 3 clinics in Philadelphia. This design was chosen to allow us to quantify "exchange rates" between different characteristics of a prediction model. We provided previously validated educational modules to explain model attributes of sensitivity, specificity, confidence intervals (CI), and time horizons. Patients reported their interest in using prediction models themselves or having their physicians use them. Patients then chose between 2 hypothetical prediction models each containing varying levels of the 4 attributes across 12 tasks. Results. We completed interviews with 200 patients, among whom 95% correctly chose a strictly dominant model in an internal validity check. Patients' interest in predictive information was high for use by themselves (n = 169, 85%) and by their physicians (n = 184, 92%). Interest in maximizing sensitivity and specificity were similar (0.88 percentage points of specificity equivalent to 1 point of sensitivity, 95% CI 0.72 to 1.05). Patients were willing to accept a reduction of 6.10 months (95% CI 3.66 to 8.54) in the predictive time horizon for a 1% increase in specificity. Discussion. Patients with chronic lung disease can articulate their preferences for the characteristics of hypothetical mortality prediction models and are highly interested in using such models as part of their care. Just as clinical care should become more patient centered, so should the characteristics of predictive models used to guide that care.