A systematic review of studies concerning observer agreement during medical specialist diagnosis using videoconferencing

Reliability and usability of telemedicine evaluations for facial dystonia

This study investigated telemedicine reliability and usability in evaluating facial dystonia grading and treatment complications. Eighty-two telemedicine recordings from 43 adults with blepharospasm (12, 28%) and hemifacial spasm (31, 72%) were obtained (mean age 64.5 ± 9.3 years, 32 females [64%]). Two recorded in-hospital telemedicine visits were arranged with in-person visits at baseline and 4-6 weeks. After 8 weeks, neuro-ophthalmologists who performed the in-person visits re-evaluated the telemedicine video records. Intra-rater agreements in assessing spasm gradings were moderate (severity: kappa = 0.42, 95% confidence interval [CI] 0.21-0.62; frequency: kappa = 0.41, 95% CI 0.21-0.61) with substantial agreement in detecting lagophthalmos (kappa = 0.61, 95% CI 0.36-0.86). Adding symptoms to signs increased sensitivity and negative predictive value (NPV) in detecting lagophthalmos (67%-100% and 94%-100%) and drooping lips (38%-75% and 94%-96%), respectively. Thai version Telehealth Usability Questionnaire showed high mean usability score of 6.5 (SD 0.8) out of 7. Telemedicine could further be developed as an alternative platform to evaluate facial dystonia.

A Systematic Review of Physical Examination Components Adapted for Telemedicine

Introduction: The COVID-19 pandemic ushered in a rapid, transformative adoption of telemedicine to maintain patient access to care. As clinicians made the shift from in-person to virtual practice, they faced a paucity of established and reliable clinical examination standards for virtual care settings. In this systematic review, we summarize the accuracy and reliability of virtual assessments compared with traditional in-person examination tools. Methods: We searched PubMed, Embase, Web of Science, and CINAHL from inception through September 2019 and included additional studies from handsearching of reference lists. We included studies that compared synchronous video (except allowing for audio-only modality for cardiopulmonary exams) with in-person clinical assessments of patients in various settings. We excluded behavioral health and dermatological assessments. Two investigators abstracted data using a predefined protocol. Results: A total of 64 studies were included and categorized into 5 clinical domains: neurological (N = 41), HEENT (head, eyes, ears, nose, and throat; N = 5), cardiopulmonary (N = 5), musculoskeletal (N = 8), and assessment of critically ill patients (N = 5). The cognitive assessment within the neurological exam was by far the most studied (N = 19) with the Mini-Mental Status Exam found to be highly reliable in multiple settings. Most studies showed relatively good reliability of the virtual assessment, although sample sizes were often small (<50 participants). Conclusions: Overall, virtual assessments performed similarly to in-person exam components for diagnostic accuracy but had a wide range of interrater reliability. The high heterogeneity in population, setting, and outcomes reported across studies render it difficult to draw broad conclusions on the most effective exam components to adopt into clinical practice. Further work is needed to identify virtual exam components that improve diagnostic accuracy.

Use of telemedicine for rheumatology practice in Queensland, Australia: Experiences before and during the COVID-19 pandemic

BACKGROUND

In Australia there is a shortage of rheumatologists potentially translating to poorer outcomes. A possible solution in this setting is telemedicine (TM).

AIM

The aim of this study was to examine the utilisation and provider perceptions of TM in rheumatology in Queensland and explore the challenges faced when using TM before and during COVID-19.

METHODS

A sequential mixed-methods study design was used. Rheumatologists completed a questionnaire on demographics, clinical practice, TM uptake, models of care and clinician perceptions of TM. The qualitative phase utilised purposeful sampling of active users of TM through in-depth semi-structured interviews.

RESULTS

Thirty rheumatologists participated with 76.7% identifying as active TM users. Use of TM was limited prior to COVID-19 with 80.9% seeing less than 5 patients per week. Patient populations served by TM included capital city (53.3%), regional (63.3%) and rural/remote (23.3%). Most rheumatologists prescribed conventional or biologic disease modifying agents (90% and 55%) through TM consultations. Barriers to TM use included low confidence in joint assessments, limited distribution of technology, access to administrative and peripheral clinical staff and lack of financial incentives. During the COVID-19 pandemic, a significant expansion of TM via telephone calls occurred and rheumatologists reported low confidence and satisfaction with this model.

CONCLUSIONS

Familiarity with TM exists in this rheumatologist cohort, however its use in routine practise is limited due to multiple barriers. The COVID 19 pandemic highlighted low confidence in telephone calls as a form of TM underlining the need for appropriate TM models of care for rheumatology practice. This article is protected by copyright. All rights reserved.

Best Practices for Virtual Care, a Consensus Statement from the Canadian Rheumatology Association

Objective

To develop best practice statements for the provision of virtual care in adult and pediatric rheumatology for the Canadian Rheumatology Association's (CRA) Telehealth Working Group (TWG).

Methods

Four members of the TWG representing adult, pediatric, university-based and community rheumatology practices defined the scope of the project. A rapid literature review was conducted between April-May 2021 of existing systematic reviews, policy documents and published literature and abstracts on the topic. The review informed a candidate set of 7 statements and a supporting document. The statements were submitted to a 3-round (R) modified Delphi process with 22 panelists recruited through the CRA and patient advocacy organizations. Panelists rated the importance and feasibility of the statements on a Likert scale of 1-9. Statements with final median ratings between 7-9 with no disagreement were retained in the final set.

Results

21 (95%) panelists participated in R1, 15 (71%) in R2 and 18 (82%) in R3. All but one statement met inclusion criteria during R1. Revisions were made to 5/7 statements following R2 and an additional statement was added. All statements met inclusion criteria following R3. The statements addressed the following themes in the provision of virtual care: adherence to existing standards and regulations, appropriateness, consent, physical examination, patient-reported outcomes, use in addition to in-person visits, and complex co-management of disease.

Conclusion

The best practice statements represent a starting point for advancing virtual care in rheumatology. Future educational efforts to help implement these best practices and research to address identified knowledge gaps are planned.

A Virtual Ward Model of Care for Patients With COVID-19: Retrospective Single-Center Clinical Study

BACKGROUND

COVID-19 has necessitated the implementation of innovative health care models in preparation for an influx of patients. A virtual ward model delivers clinical care remotely to patients in isolation. We report on an Australian cohort of patients with COVID-19 treated in a virtual ward.

OBJECTIVE

The aim of this study was to describe and evaluate the safety and efficacy of a virtual ward model of care for an Australian cohort of patients with COVID-19.

METHODS

Retrospective clinical assessment was performed for 223 patients with confirmed COVID-19 treated in a virtual ward in Brisbane, Australia, from March 25 to May 15, 2020. Statistical analysis was performed for variables associated with the length of stay and hospitalization.

RESULTS

Of 223 patients, 205 (92%) recovered without the need for escalation to hospital care. The median length of stay in the virtual ward was 8 days (range 1-44 days). In total, 18 (8%) patients were referred to hospital, of which 6 (33.3%) were discharged after assessment at the emergency department. Furthermore, 12 (5.4%) patients were admitted to hospital, of which 4 (33.3%) required supplemental oxygen and 2 (16.7%) required mechanical ventilation. No deaths were recorded. Factors associated with escalation to hospital care were the following: hypertension (odds ratio [OR] 3.6, 95% CI 1.28-9.87; P=.01), sputum production (OR 5.2, 95% CI 1.74-15.49; P=.001), and arthralgia (OR 3.8, 95% CI 1.21-11.71; P=.02) at illness onset and a polymerase chain reaction cycle threshold of ≤20 on a diagnostic nasopharyngeal swab (OR 5.0, 95% CI 1.25-19.63; P=.02).

CONCLUSIONS

Our results suggest that a virtual ward model of care to treat patients with COVID-19 is safe and efficacious, and only a small number of patients would potentially require escalation to hospital care. Further studies are required to validate this model of care.

Telerheumatology: A Systematic Review

OBJECTIVE

To identify and summarize the published and gray literature on the use of telemedicine for the diagnosis and management of inflammatory and/or autoimmune rheumatic disease.

METHODS

We performed a registered systematic search (CRD42015025382) for studies using MEDLINE (1946 to July 2015), Embase (1974 to July 2015), Web of Science (1900 to July 2015), and Scopus (1946 to July 2015) databases. We included studies that demonstrated the use of telemedicine for diagnosis and/or management of inflammatory/autoimmune rheumatic disease. Following data extraction, we performed a descriptive analysis.

RESULTS

Our literature search identified 1,468 potentially eligible studies. Of these studies, 20 were ultimately included in this review. Studies varied significantly in publication type, quality of evidence, and the reporting of methods. Most demonstrated a high risk of bias. Rheumatoid arthritis was the most commonly studied rheumatic disease (42% of patients). Studies demonstrated conflicting results regarding the effectiveness of telemedicine (18 found it effective, 1 found it effective but possibly harmful, and 1 found it ineffective). A limited number of studies included some component of a cost analysis (n = 6; 16% of patients); all of these found telemedicine to be cost-effective.

CONCLUSION

Studies identified by this systematic review generally found telemedicine to be effective for the diagnosis and management of autoimmune/inflammatory rheumatic disease; however, there is limited evidence to support this conclusion. Further studies are needed to determine the best uses of telemedicine for the diagnosis and management of these conditions.

Reliability of an Online Geriatric Assessment Procedure Using the interRAI Acute Care Assessment System

OBJECTIVES

To determine whether geriatric triage decisions made using a comprehensive geriatric assessment (CGA) performed online are less reliable than face-to-face (FTF) decisions.

DESIGN

Multisite noninferiority prospective cohort study. Two specialist geriatricians assessed individuals sequentially referred for an acute care geriatric consultation. Participants were allocated to one FTF assessment and an additional assessment (FTF or online (OL)), creating two groups-two FTF (FTF-FTF, n = 81) or online and FTF (OL-FTF, n = 85).

SETTING

Three acute care public hospitals in two Australian states.

PARTICIPANTS

Admitted individuals referred for CGA.

INTERVENTION

Nurse-administered CGA, based on the interRAI Acute Care assessment system accessed online and other online clinical data such as pathology results and imaging enabling geriatricians to review participants' information and provide input into their care from a distance.

MEASUREMENTS

The primary decision subjected to this analysis was referral for permanent residential care. Geriatricians also recorded recommendations for referrals and variations for medication management and judgment regarding prognosis at discharge and after 3 months.

RESULTS

Overall percentage agreement was 88% (n = 71) for the FTF-FTF group and 91% (n = 77) for the OL-FTF group. The difference in agreement between the FTF-FTF and OL-FTF groups was -3%, indicating that there was no difference between the methods of assessment. Judgements made regarding diagnoses of geriatric syndromes, medication management, and prognosis (with regard to hospital outcome and location at 3 months) were found to be equally reliable in each mode of consultation.

CONCLUSION

Geriatric assessment performed online using a nurse-administered structured CGA system was no less reliable than conventional assessment in making clinical triage decisions.

Telemedicine for Specialist Geriatric Care in Small Rural Hospitals: Preliminary Data

Small rural hospitals admit and manage older adults who, in city hospitals, would usually be offered geriatrician-supported comprehensive geriatric assessment and coordinated subacute care if required. Distance and diseconomies of scale prohibit access to the conventional in-person approach. A telegeriatric service model involving a geriatrician consulting remotely using wireless, mobile, high-definition videoconferencing; a trained host nurse at the rural site; structured geriatric assessment configured on a web-based clinical decision support system; routine weekly virtual rounds; and support from a local multidisciplinary team was established to overcome these barriers. This was a prospective observational study to examine the feasibility and sustainability of the model. Patient characteristics were recorded using the interRAI Acute Care assessment system. Usage patterns were derived from health service data sets and a service statistics database. Patients had characteristics that are consistent with characteristics of individuals typically referred for geriatric assessment. Overall, 53% of patients had cognitive impairment, 75% had limitations with activities of daily living, and the average Frailty Index was 0.44 ± 0.12. Stable patterns of consultation occurred within 6 months of start-up and continued uninterrupted for the remainder of the 24-month observation period. The estimated overall rate of initial consultation was 1.83 cases per occupied bed per year and 2.66 review cases per occupied bed per year. The findings indicate that the model was feasible and was sustained throughout and beyond the study period. This telegeriatric service model appears suitable for use in small rural hospitals.

When New Technology is an Answer for Old Problems: The Use of Videoconferencing in Cognitive Aging Assessment

People living with cognitive impairment and dementia require regular screening of their symptomatology and needs. Brief cognitive screening is crucial for assessing these conditions. Such screening can give health professionals a snapshot of the patient's cognitive abilities and help in their monitoring and adaptation. An appropriate administration of brief cognitive screening using telemedicine technology, such as videoconference, can improve access to care and treatment planning. Moreover, the burden that rural and underserved communities often suffer because of limited access to specialty mental health services is also of concern. Herein, in this review, we describe the existing and relevant research regarding the administration of neuropsychological instruments via videoconferencing, and suggest methodological improvements for further studies. To date, only a handful of scientific studies have been published documenting the feasibility and acceptability of videoconferencing among older people and health professionals. Varying in their assessment tools, the studies demonstrate the reliability of cognitive testing and clinical diagnosis of dementia via videoconferencing, as noted by the associations between results of several cognitive tests using it as compared to face-to-face interviews. Further research is required to replicate findings and bring new solutions for their adaptation to a wide range of individuals, health professionals, areas of practice and settings.

Usefulness of telepsychiatry: A critical evaluation of videoconferencing-based approaches

Telepsychiatry, i.e., the use of information and communication technologies to provide psychiatric services from a distance, has been around for more than half a century now. Research over this period has shown that videoconferencing-based telepsychiatry is an enabling and empowering form of service delivery, which promotes equality of access, and high levels of satisfaction among patients. The range of services offered by videoconferencing-based telepsychiatry, potential users and points of delivery of such services are theoretically limitless. Telepsychiatry has both clinical utility and non-clinical uses such as administrative, learning and research applications. A large body of accumulated evidence indicates that videoconferencing-based telepsychiatric assessments are reliable, and clinical outcomes of telepsychiatric interventions are comparable to conventional treatment among diverse patient populations, ages and diagnostic groups, and on a wide range of measures. However, on many aspects of effectiveness, the evidence base is still relatively limited and often compromised by methodological problems. The lack of cost-effectiveness data in particular, is a major hindrance, raising doubts about the continued viability of telepsychiatric services. Added to this are the vagaries of technology, negative views among clinicians, poor uptake by providers, and several legal, ethical and administrative barriers. These hamper the widespread implementation of telepsychiatry and its integration with routine care. Though further advances in technology and research are expected to solve many of these problems, the way forward would be to promote telepsychiatry as an adjunct to conventional care, and to develop hybrid models, which incorporate both traditional and telepsychiatric forms of mental health-care.

Telerheumatology: an idea whose time has come

Australia is a vast country with one-third of the population living outside capital cities. Providing specialist rheumatologist services to regional, rural and remote Australians has generally required expensive and time-consuming travel for the patient and/or specialist. As a result, access to specialist care for remote Australians is poor. Rheumatoid arthritis is a common disease, but like many rheumatic diseases, it is complex to treat. Time-dependent joint damage and disability occur unless best evidence care is implemented. The relatively poor access to rheumatologist care allotted to nonmetropolitan Australians therefore represents a significant cause of potentially preventable disability in Australia. Telehealth has the potential to improve access to specialist rheumatologists for patients with rheumatoid arthritis and other rheumatic diseases, thereby decreasing the burden of disability caused by these diseases. Advances in videoconferencing technology, the national broadband rollout and recent Federal government financial incentives have led to a heightened interest in exploring the use of this technology in Australian rheumatology practice. This review summarises the current evidence base, outlines telehealth's strengths and weaknesses in managing rheumatic disease, and discusses the technological, medicolegal and financial aspects of this model of care. A mixed model offering both face-to-face and virtual consultations appears to be the best option, as it can overcome the barriers to accessing care posed by distance while also mitigating the risks of virtual consultation.

The diagnostic accuracy of telegeriatrics for the diagnosis of dementia via video conferencing

INTRODUCTION

The suitability of video conferencing (VC) technology for clinical purposes relevant to geriatric medicine is still being established. This project aimed to determine the validity of the diagnosis of dementia via VC.

METHODS

This was a multisite, noninferiority, prospective cohort study. Patients, aged 50 years and older, referred by their primary care physician for cognitive assessment, were assessed at 4 memory disorder clinics. All patients were assessed independently by 2 specialist physicians. They were allocated one face-to-face (FTF) assessment (Reference standard--usual clinical practice) and an additional assessment (either usual FTF assessment or a VC assessment) on the same day. Each specialist physician had access to the patient chart and the results of a battery of standardized cognitive assessments administered FTF by the clinic nurse. Percentage agreement (P(0)) and the weighted kappa statistic with linear weight (K(w)) were used to assess inter-rater reliability across the 2 study groups on the diagnosis of dementia (cognition normal, impaired, or demented).

RESULTS

The 205 patients were allocated to group: Videoconference (n = 100) or Standard practice (n = 105); 106 were men. The average age was 76 (SD 9, 51-95) and the average Standardized Mini-Mental State Examination Score was 23.9 (SD 4.7, 9-30). Agreement for the Videoconference group (P(0)= 0.71; K(w) = 0.52; P < .0001) and agreement for the Standard Practice group (P(0)= 0.70; K(w) = 0.50; P < .0001) were both statistically significant (P < .05). The summary kappa statistic of 0.51 (P = .84) indicated that VC was not inferior to FTF assessment.

CONCLUSIONS

Previous studies have shown that preliminary standardized assessment tools can be reliably administered and scored via VC. This study focused on the geriatric assessment component of the interview (interpretation of standardized assessments, taking a history and formulating a diagnosis by medical specialist) and identified high levels of agreement for diagnosing dementia. A model of service incorporating either local or remote administered standardized assessments, and remote specialist assessment, is a reliable process for enabling the diagnosis of dementia for isolated older adults.

Agreement on Web-based Diagnoses and Severity of Mental Health Problems in Norwegian Child and Adolescent Mental Health Services

OBJECTIVE

This study examined the agreement between diagnoses and severity ratings assigned by clinicians using a structured web-based interview within a child and adolescent mental health outpatient setting.

METHOD

Information on 100 youths was obtained from multiple informants through a web-based Development and Well-Being Assessment (DAWBA). Based on this information, four experienced clinicians independently diagnosed (according to the International Classification of Diseases Revision 10) and rated the severity of mental health problems according to the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) and the Children's Global Assessment Scale (C-GAS).

RESULTS

Agreement for diagnosis was κ=0.69-0.82. Intra-class correlation for single measures was 0.78 for HoNOSCA and 0.74 for C-GAS, and 0.93 and 0.92, respectively for average measures.

CONCLUSIONS

Agreement was good to excellent for all diagnostic categories. Agreement for severity was moderate, but improved to substantial when the average of the ratings given by all clinicians was considered. Therefore, we conclude that experienced clinicians can assign reliable diagnoses and assess severity based on DAWBA data collected online.