Machine learning-based improvement of an online rheumatology referral and triage system

Prioritising Appointments by Telephone Interview: Duration from Symptom Onset to Appointment Request Predicts Likelihood of Inflammatory Rheumatic Disease

Background: This study aims to determine the rate of inflammatory rheumatic diseases (IRDs) in a cohort of initial referrals and the efficacy of prioritising appointments to the early arthritis clinic (EAC) based on symptom duration. Methods: In the present study, we used algorithm-based telephone triage to assign routine care appointments according to the time between symptom onset and request for an appointment (cut-off criterion: 6 months). This retrospective, monocentric analysis evaluated the effectiveness of our triage in identifying patients with IRDs as a function of the assigned appointment category (elective, EAC, or emergency appointment). Results: A total of 1407 patients were included in the study (34.7% male; 65.3% female). Of the 1407 patients evaluated, 361 (25.7%) presented with IRD. There were significant differences in the frequency of inflammatory diagnoses between appointment categories (p < 0.001): elective 13.8%, EAC 32.9%, and emergency 45.9%. The sample without the emergency category included a total of 1222 patients. The classification into "inflammatory" or "non-inflammatory" in this subsample was as follows: Sensitivity was 37.7%, and specificity was 92.6%. The positive predictive value (PPV) was 59.8%, and the negative predictive value (NPV) was 83.6%. Overall, 80.2% of patients were correctly assigned using the appointment category and C-reactive protein (CRP). Conclusions: The algorithm-based triage system presented here, which focuses on the time between symptom onset and request for an appointment, allows for the prioritisation of appointments in favour of patients with IRDs and thus earlier initiation of therapy.

Diagnostic Accuracy of a Mobile AI-Based Symptom Checker and a Web-Based Self-Referral Tool in Rheumatology: Multicenter Randomized Controlled Trial

BACKGROUND

The diagnosis of inflammatory rheumatic diseases (IRDs) is often delayed due to unspecific symptoms and a shortage of rheumatologists. Digital diagnostic decision support systems (DDSSs) have the potential to expedite diagnosis and help patients navigate the health care system more efficiently.

OBJECTIVE

The aim of this study was to assess the diagnostic accuracy of a mobile artificial intelligence (AI)-based symptom checker (Ada) and a web-based self-referral tool (Rheport) regarding IRDs.

METHODS

A prospective, multicenter, open-label, crossover randomized controlled trial was conducted with patients newly presenting to 3 rheumatology centers. Participants were randomly assigned to complete a symptom assessment using either Ada or Rheport. The primary outcome was the correct identification of IRDs by the DDSSs, defined as the presence of any IRD in the list of suggested diagnoses by Ada or achieving a prespecified threshold score with Rheport. The gold standard was the diagnosis made by rheumatologists.

RESULTS

A total of 600 patients were included, among whom 214 (35.7%) were diagnosed with an IRD. Most frequent IRD was rheumatoid arthritis with 69 (11.5%) patients. Rheport's disease suggestion and Ada's top 1 (D1) and top 5 (D5) disease suggestions demonstrated overall diagnostic accuracies of 52%, 63%, and 58%, respectively, for IRDs. Rheport showed a sensitivity of 62% and a specificity of 47% for IRDs. Ada's D1 and D5 disease suggestions showed a sensitivity of 52% and 66%, respectively, and a specificity of 68% and 54%, respectively, concerning IRDs. Ada's diagnostic accuracy regarding individual diagnoses was heterogenous, and Ada performed considerably better in identifying rheumatoid arthritis in comparison to other diagnoses (D1: 42%; D5: 64%). The Cohen κ statistic of Rheport for agreement on any rheumatic disease diagnosis with Ada D1 was 0.15 (95% CI 0.08-0.18) and with Ada D5 was 0.08 (95% CI 0.00-0.16), indicating poor agreement for the presence of any rheumatic disease between the 2 DDSSs.

CONCLUSIONS

To our knowledge, this is the largest comparative DDSS trial with actual use of DDSSs by patients. The diagnostic accuracies of both DDSSs for IRDs were not promising in this high-prevalence patient population. DDSSs may lead to a misuse of scarce health care resources. Our results underscore the need for stringent regulation and drastic improvements to ensure the safety and efficacy of DDSSs.

TRIAL REGISTRATION

German Register of Clinical Trials DRKS00017642; https://drks.de/search/en/trial/DRKS00017642.

Machine learning models for classifying non-specific neck pain using craniocervical posture and movement

OBJECTIVE

Physical therapists and clinicians commonly confirm craniocervical posture (CCP), cervical retraction, and craniocervical flexion as screening tests because they contribute to non-specific neck pain (NSNP). We compared the predictive performance of statistical machine learning (ML) models for classifying individuals with and without NSNP using datasets containing CCP and cervical kinematics during pro- and retraction (CKdPR).

DESIGN

Exploratory, cross-sectional design.

SETTING AND PARTICIPANTS

In total, 773 public service office workers (PSOWs) were screened for eligibility (NSNP, 441; without NSNP, 332).

METHODS

We set up five datasets (CCP, cervical kinematics during the protraction, cervical kinematics during the retraction, CKdPR and combination of the CCP and CKdPR). Four ML algorithms-random forest, logistic regression, Extreme Gradient boosting, and support vector machine-were trained.

MAIN OUTCOME MEASURES

Model performance were assessed using area under the curve (AUC), accuracy, precision, recall and F1-score. To interpret the predictions, we used Feature permutation importance and SHapley Additive explanation values.

RESULTS

The random forest model in the CKdPR dataset classified PSOWs with and without NSNP and achieved the best AUC among the five datasets using the test data (AUC, 0.892 [good]; F1, 0.832). The random forest model in the CCP dataset had the worst AUC among the five datasets using the test data [AUC, 0.738 (fair); F1, 0.715].

CONCLUSION

ML performance was higher for the CKdPR dataset than for the CCP dataset, suggesting that ML algorithms are more suitable than classical statistical methods for developing robust models for classifying PSOWs with and without NSNP.

Prediction of the acceptance of telemedicine among rheumatic patients: a machine learning-powered secondary analysis of German survey data

Telemedicine (TM) has augmented healthcare by enabling remote consultations, diagnosis, treatment, and monitoring of patients, thereby improving healthcare access and patient outcomes. However, successful adoption of TM depends on user acceptance, which is influenced by technical, socioeconomic, and health-related factors. Leveraging machine learning (ML) to accurately predict these adoption factors can greatly contribute to the effective utilization of TM in healthcare. The objective of the study was to compare 12 ML algorithms for predicting willingness to use TM (TM try) among patients with rheumatic and musculoskeletal diseases (RMDs) and identify key contributing features. We conducted a secondary analysis of RMD patient data from a German nationwide cross-sectional survey. Twelve ML algorithms, including logistic regression, random forest, extreme gradient boosting (XGBoost), and neural network (deep learning) were tested on a subset of the dataset, with the inclusion of only RMD patients who answered "yes" or "no" to TM try. Nested cross-validation was used for each model. The best-performing model was selected based on area under the receiver operator characteristic (AUROC). For the best-performing model, a multinomial/multiclass ML approach was undertaken with the consideration of the three following classes: "yes", "no", "do not know/not answered". Both one-vs-one and one-vs-rest strategies were considered. The feature importance was investigated using Shapley additive explanation (SHAP). A total of 438 RMD patients were included, with 26.5% of them willing to try TM, 40.6% not willing, and 32.9% undecided (missing answer or "do not know answer"). This dataset was used to train and test ML models. The mean accuracy of the 12 ML models ranged from 0.69 to 0.83, while the mean AUROC ranged from 0.79 to 0.90. The XGBoost model produced better results compared with the other models, with a sensitivity of 70%, specificity of 91% and positive predictive value of 84%. The most important predictors of TM try were the possibility that TM services were offered by a rheumatologist, prior TM knowledge, age, self-reported health status, Internet access at home and type of RMD diseases. For instance, for the yes vs. no classification, not wishing that TM services were offered by a rheumatologist, self-reporting a bad health status and being aged 60-69 years directed the model toward not wanting to try TM. By contrast, having Internet access at home and wishing that TM services were offered by a rheumatologist directed toward TM try. Our findings have significant implications for primary care, in particular for healthcare professionals aiming to implement TM effectively in their clinical routine. By understanding the key factors influencing patients' acceptance of TM, such as their expressed desire for TM services provided by a rheumatologist, self-reported health status, availability of home Internet access, and age, healthcare professionals can tailor their strategies to maximize the adoption and utilization of TM, ultimately improving healthcare outcomes for RMD patients. Our findings are of high interest for both clinical and medical teaching practice to fit changing health needs caused by the growing number of complex and chronically ill patients.

Stepwise asynchronous telehealth assessment of patients with suspected axial spondyloarthritis: results from a pilot study

Patients with axial spondyloarthritis (axSpA) suffer from one of the longest diagnostic delays among all rheumatic diseases. Telemedicine (TM) may reduce this diagnostic delay by providing easy access to care. Diagnostic rheumatology telehealth studies are scarce and largely limited to traditional synchronous approaches such as resource-intensive video and telephone consultations. The aim of this study was to investigate a stepwise asynchronous telemedicine-based diagnostic approach in patients with suspected axSpA. Patients with suspected axSpA completed a fully automated digital symptom assessment using two symptom checkers (SC) (bechterew-check and Ada). Secondly, a hybrid stepwise asynchronous TM approach was investigated. Three physicians and two medical students were given sequential access to SC symptom reports, laboratory and imaging results. After each step, participants had to state if axSpA was present or not (yes/no) and had to rate their perceived decision confidence. Results were compared to the final diagnosis of the treating rheumatologist. 17 (47.2%) of 36 included patients were diagnosed with axSpA. Diagnostic accuracy of bechterew-check, Ada, TM students and TM physicians was 47.2%, 58.3%, 76.4% and 88.9% respectively. Access to imaging results significantly increased sensitivity of TM-physicians (p < 0.05). Mean diagnostic confidence of false axSpA classification was not significantly lower compared to correct axSpA classification for both students and physicians. This study underpins the potential of asynchronous physician-based telemedicine for patients with suspected axSpA. Similarly, the results highlight the need for sufficient information, especially imaging results to ensure a correct diagnosis. Further studies are needed to investigate other rheumatic diseases and telediagnostic approaches.

"The Simpler, the Better." A Qualitative Study on Digital Health Transformation in Early Adopter Rheumatology Outpatient Clinics

Workforce shortage and the increasing burden of rheumatic and musculoskeletal diseases lead to extreme time constraints in rheumatology outpatient care. Digital services promise to facilitate care by relieving employees and unleash new capacities. This study aims to explore the perspectives of early adopter health care professionals (HCP) on digital transformation in outpatient rheumatology. In-depth qualitative interviews were conducted with rheumatology nurses and physicians in 3 German rheumatology outpatient clinics, each characterized by an advanced level of digital adaption. Qualitative data were subsequently analyzed using deductive-inductive qualitative content analysis. Interviews with 11 rheumatology nurses and 5 rheumatologists were completed. Three key themes emerged from the qualitative analysis: (i) Digital transformation of care; (ii) impact of digital transformation on health care delivery; and (iii) perceived drivers of successful digitalization. The interviews revealed that digital technologies are widely used throughout the complete patient pathway. Digitalization enables more continuity and flexibility in rheumatology care. Patient information can be electronically obtained in a standardized manner prior to planned visits, enabling an informed consultation and more time for in-depth patient discussion. Although digitalization restructures work, it can also increase the current workload. Improved accessibility for patient calls leads to more work for HCP. Important drivers of successful digital technology implementation are low-threshold and interoperable services, a medical team that is interested and educated in eHealth, and comprehensive patient information and onboarding. Digital transformation is increasingly redefining rheumatology care. While accelerating communication and workflows, improved service accessibility leads to more work for HCP.

[Digital rheumatology]

Chronic inflammatory rheumatic diseases mostly run an undulating course and with unspecific symptoms. The initial clarification and timely initiation of treatment are challenging, which is additionally exacerbated by the lack of specialized physicians. Digital approaches, including artificial intelligence (AI), should be of assistance and enable an improved, personalized and needs-based treatment; however, the evidence is currently still very limited. This article provides a compact overview of the current state of digital rheumatology.

Diagnostic delay in axial spondylarthritis: A lost battle?

Diagnostic delay in axial spondylarthritis (axSpA) remains an unacceptable worldwide problem; with evidence suggesting significant detrimental impact both clinically on the individual, and economically on society. There is therefore, a need for global action across various healthcare professions that come into contact with patients living, and suffering, with undiagnosed axSpA. Recent estimates of the median diagnostic delay suggest that globally, individuals with axSpA wait between 2 and 6 years for a diagnosis - revealing a clear benchmark for improvement. This timespan presents a window of opportunity for earlier diagnosis and intervention, which will likely improve patient outcomes. This review describes the current diagnostic delay as estimated across countries and over time, before presenting evidence from published strategies that may be implemented to improve this delay across primary and secondary care, including for specialties treating extra-musculoskeletal manifestations of axSpA (ophthalmology, gastroenterology, dermatology). Ongoing campaigns tackling delayed diagnosis in axSpA are also highlighted.