Telehealth Made EASY : Understanding Provider Perceptions of Telehealth Appropriateness in Outpatient Rheumatology Encounters

Evaluating Factors Associated With Telehealth Appropriateness in Outpatient Rheumatoid Arthritis Encounters Using the Encounter Appropriateness Score for You (EASY)

OBJECTIVE

Telehealth has been proposed as a safe and effective alternative to in-person care for rheumatoid arthritis (RA). The purpose of this study was to evaluate factors associated with telehealth appropriateness in outpatient RA encounters.

METHODS

A prospective cohort study (January 1, 2021, to August 31, 2021) was conducted using electronic health record data from outpatient RA encounters in a single academic rheumatology practice. Rheumatology providers rated the telehealth appropriateness of their own encounters using the Encounter Appropriateness Score for You (EASY) immediately following each encounter. Robust Poisson regression with generalized estimating equations modeling was used to evaluate the association of telehealth appropriateness with patient demographics, RA clinical characteristics, comorbid noninflammatory causes of joint pain, previous and current encounter characteristics, and provider characteristics.

RESULTS

During the study period, 1823 outpatient encounters with 1177 unique patients with RA received an EASY score from 25 rheumatology providers. In the final multivariate model, factors associated with increased telehealth appropriateness included higher average provider preference for telehealth in prior encounters (relative risk [RR] 1.26, 95% CI 1.21-1.31), telehealth as the current encounter modality (RR 2.27, 95% CI 1.95-2.64), and increased patient age (RR 1.05, 95% CI 1.01-1.09). Factors associated with decreased telehealth appropriateness included moderate (RR 0.81, 95% CI 0.68-0.96) and high (RR 0.57, 95% CI 0.46-0.70) RA disease activity and if the previous encounters were conducted by telehealth (RR 0.83, 95% CI 0.73-0.95).

CONCLUSION

In this study, telehealth appropriateness was most associated with provider preference, the current and previous encounter modality, and RA disease activity. Other factors like patient demographics, RA medications, and comorbid noninflammatory causes of joint pain were not associated with telehealth appropriateness.

Evaluating Provider Perceptions of Telehealth Utility in Outpatient Rheumatology Telehealth Encounters

OBJECTIVE

This study aims to explore the factors associated with rheumatology providers' perceptions of telehealth utility in real-world telehealth encounters.

METHODS

From September 14, 2020 to January 31, 2021, 6 providers at an academic medical center rated their telehealth visits according to perceived utility in making treatment decisions using the following Telehealth Utility Score (TUS) (1 = very low utility to 5 = very high utility). Modified Poisson regression models were used to assess the association between TUS scores and encounter diagnoses, disease activity measures, and immunomodulatory therapy changes during the encounter.

RESULTS

A total of 481 telehealth encounters were examined, of which 191 (39.7%) were rated as "low telehealth utility" (TUS 1-3) and 290 (60.3%) were rated as "high telehealth utility" (TUS 4-5). Encounters with a diagnosis of inflammatory arthritis were significantly less likely to be rated as high telehealth utility (adjusted relative risk [aRR], 0.8061; p = 0.004), especially in those with a concurrent noninflammatory musculoskeletal diagnosis (aRR, 0.54; p = 0.006). Other factors significantly associated with low telehealth utility included higher disease activity according to current and prior RAPID3 scores (aRR, 0.87 and aRR, 0.89, respectively; p < 0.001) and provider global scores (aRR, 0.83; p < 0.001), as well as an increase in immunomodulatory therapy (aRR, 0.70; p = 0.015).

CONCLUSIONS

Provider perceptions of telehealth utility in real-world encounters are significantly associated with patient diagnoses, current and prior disease activity, and the need for changes in immunomodulatory therapy. These findings inform efforts to optimize the appropriate utilization of telehealth in rheumatology.

Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare

OBJECTIVE

The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion.

MATERIALS AND METHODS

Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution.

RESULTS

An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center.

DISCUSSION

By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution.

CONCLUSIONS

We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

Encounter Appropriateness Score for You Model: Development and Pilot Implementation of a Predictive Model to Identify Visits Appropriate for Telehealth in Rheumatology

OBJECTIVE

We aimed to develop a decision-making tool to predict telehealth appropriateness for future rheumatology visits and expand telehealth care access.

METHODS

The model was developed using the Encounter Appropriateness Score for You (EASY) and electronic health record data at a single academic rheumatology practice from January 1, 2021, to December 31, 2021. The EASY model is a logistic regression model that includes encounter characteristics, patient sociodemographic and clinical characteristics, and provider characteristics. The goal of pilot implementation was to determine if model recommendations align with provider preferences and influence telehealth scheduling. Four providers were presented with future encounters that the model identified as candidates for a change in encounter modality (true changes), along with an equal number of artificial (false) recommendations. Providers and patients could accept or reject proposed changes.

RESULTS

The model performs well, with an area under the curve from 0.831 to 0.855 in 21,679 encounters across multiple validation sets. Covariates that contributed most to model performance were provider preference for and frequency of telehealth encounters. Other significant contributors included encounter characteristics (current scheduled encounter modality) and patient factors (age, Routine Assessment of Patient Index Data 3 scores, diagnoses, and medications). The pilot included 201 encounters. Providers were more likely to agree with true versus artificial recommendations (Cohen's κ = 0.45, P < 0.001), and the model increased the number of appropriate telehealth visits.

CONCLUSION

The EASY model accurately identifies future visits that are appropriate for telehealth. This tool can support shared decision-making between patients and providers in deciding the most appropriate follow-up encounter modality.