Self-triage for acute primary care via a smartphone application: Practical, safe and efficient?

Outcome After Self-Triage App Referral in Urgent Direct-to-Consumer Telemedicine Encounter

Background: The quantification of self-triage effectiveness, guided by mobile applications, in urgent direct-to-consumer telemedicine (TM) encounters requires further investigation. The objective of this study was to evaluate the outcomes of referral guidance provided by a symptom-based self-management mobile application decision algorithm in the context of remote urgent care assessments. Methods: An observational retrospective single-center study was conducted from May 2022 to December 2023. The inclusion criteria encompassed individuals aged >18 years old, and those spontaneously seeking virtual emergency care through the EINSTEIN CONECTA application. Patients experiencing connectivity issues, preventing completion of the encounter, were excluded. The primary outcomes included the rate of patient concurrence with the algorithm's recommendation for seeking in-person emergency care and the referral rate to face-to-face assessment among cases evaluated through TM. The application's algorithm employs scientific evidence based on symptoms to recommend referrals to emergency departments (EDs). Results: Out of 88,834 patients connected to the TM Center, self-triage obviated the need for virtual physician assessment in 53,302 (60%) encounters. A total of 35,532 patients were remotely evaluated by 316 on-duty physicians, resulting in 1,125 ICD-coded diagnoses. Among these, 21,722 (61.1%) were initially advised by self-triage to visit the ED, with subsequent medical assessment leading to in-person referrals in 6,354 (29.3%) of the evaluations. Of the 13,810 patients recommended to continue with virtual care post-self-triage, 157 (1.1%) were referred for in-person assessment. Conclusions: Self-triage effectively reduced the need for physician encounters in approximately three-fifths of TM consultations. Despite being based on scientific evidence, symptom-based referral algorithms demonstrated high sensitivity but poor correlation with physician decision-making.

Parents' Trigger Tool for Children with Medical Complexity - PAT-CMC: Development of a recognition tool for clinical deterioration at home

AIM

To develop a trigger tool for parents and lay caregivers of children with medical complexity (CMC) at home and to validate its content.

DESIGN

This was a multi-method study, using qualitative data, a Delphi method and a concept mapping approach.

METHODS

A three-round electronic Delphi was performed from December 2021 to April 2022 with a panel of 23 expert parents and 30 healthcare providers, supplemented by a preliminary qualitative exploration of children's signs of deterioration and three consensus meetings to develop the PArents' Trigger Tool for Children with Medical Complexity (PAT-CMC). Cognitive interviews with parents were performed to assess the comprehensiveness and comprehensibility of the tool. The COREQ checklist, the COSMIN guidelines and the CREDES guidelines guided the reporting respectively of the qualitative study, the development and content validity of the trigger tool and the Delphi study.

RESULTS

The PAT-CMC was developed and its content validated to recognize clinical deterioration at home. The tool consists of 7 main clusters of items: Breathing, Heart, Devices, Behaviour, Neuro-Muscular, Nutrition/Hydration and Other Concerns. A total of 23 triggers of deterioration were included and related to two recommendations for escalation of care, using a traffic light coding system.

CONCLUSION

Priority indicators of clinical deterioration of CMC were identified and integrated into a validated trigger tool designed for parents or other lay caregivers at home, to recognize signs of acute severe illness and initiate healthcare interventions.

IMPACT

The PAT-CMC was developed to guide families in recognizing signs of deterioration in CMC and has potential for initiating an early escalation of care. This tool may also be useful to support education provided by healthcare providers to families before hospital discharge.

PATIENT OR PUBLIC CONTRIBUTION

Parents of CMC were directly involved in the selection of relevant indicators of children's clinical deterioration and the development of the trigger tool. They were not involved in the design, conducting, reporting or dissemination plans of this research.

Recommendation endpoints and safety of an online self-triage for depression symptoms

INTRODUCTION

Online symptom checkers are a way to address patient concerns and potentially offload a burdened healthcare system. However, safety outcomes of self-triage are unknown, so we reviewed triage recommendations and outcomes of our institution's depression symptom checker.

METHODS

We examined endpoint recommendations and follow-up encounters seven days afterward during 2 December 2021 to 13 December 2022. Patients with an emergency department visit or hospitalization within seven days of self-triaging had a manual review of the electronic health record to determine if the visit was related to depression, suicidal ideation, or suicide attempt. Charts were reviewed for deaths within seven days of self-triage.

RESULTS

There were 287 unique encounters from 263 unique patients. In 86.1% (247/287), the endpoint was an instruction to call nurse triage; in 3.1% of encounters (9/287), instruction was to seek emergency care. Only 20.2% (58/287) followed the recommendations given. Of the 229 patients that did not follow the endpoint recommendations, 121 (52.8%) had some type of follow-up within seven days. Nearly 11% (31/287) were triaged to endpoints not requiring urgent contact and 9.1% (26/287) to an endpoint that would not need any healthcare team input. No patients died in the study period.

CONCLUSIONS

Most patients did not follow the recommendations for follow-up care although ultimately most patients did receive care within seven days. Self-triage appears to appropriately sort patients with depressed mood to emergency care. On-line self-triaging tools for depression have the potential to safely offload some work from clinic personnel.

How Virtual Triage Can Improve Patient Experience and Satisfaction: A Narrative Review and Look Forward

Objective

To complete a review of the literature on patient experience and satisfaction as relates to the potential for virtual triage (VT) or symptom checkers to enhance and enable improvements in these important health care delivery objectives.

Methods

Review and synthesis of the literature on patient experience and satisfaction as informed by emerging evidence, indicating potential for VT to favorably impact these clinical care objectives and outcomes.

Results/Conclusions

VT enhances potential clinical effectiveness through early detection and referral, can reduce avoidable care delivery due to late clinical presentation, and can divert primary care needs to more clinically appropriate outpatient settings rather than high-acuity emergency departments. Delivery of earlier and faster, more acuity level-appropriate care, as well as patient avoidance of excess care acuity (and associated cost), offer promise as contributors to improved patient experience and satisfaction. The application of digital triage as a front door to health care delivery organizations offers care engagement that can help reduce patient need to visit a medical facility for low-acuity conditions more suitable for self-care, thus avoiding unpleasant queues and reducing microbiological and other patient risks associated with visits to medical facilities. VT also offers an opportunity for providers to make patient health care experiences more personalized.

Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review

BACKGROUND

In the context of a deepening global shortage of health workers and, in particular, the COVID-19 pandemic, there is growing international interest in, and use of, online symptom checkers (OSCs). However, the evidence surrounding the triage and diagnostic accuracy of these tools remains inconclusive.

OBJECTIVE

This systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns.

METHODS

Searches were conducted in MEDLINE, Embase, CINAHL, Health Management Information Consortium (HMIC), and Web of Science, as well as the citations of the studies selected for full-text screening. We included peer-reviewed studies published in English between January 1, 2010, and February 16, 2022, with a controlled and quantitative assessment of either or both triage and diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health care professionals, as well as disease- or specialty-specific OSCs. Screening and data extraction were carried out independently by 2 reviewers for each study. We performed a descriptive narrative synthesis.

RESULTS

A total of 21,296 studies were identified, of which 14 (0.07%) were included. The included studies used clinical vignettes, medical records, or direct input by patients. Of the 14 studies, 6 (43%) reported on triage and diagnostic accuracy, 7 (50%) focused on triage accuracy, and 1 (7%) focused on diagnostic accuracy. These outcomes were assessed based on the diagnostic and triage recommendations attached to the vignette in the case of vignette studies or on those provided by nurses or general practitioners, including through face-to-face and telephone consultations. Both diagnostic accuracy and triage accuracy varied greatly among OSCs. Overall diagnostic accuracy was deemed to be low and was almost always lower than that of the comparator. Similarly, most of the studies (9/13, 69 %) showed suboptimal triage accuracy overall, with a few exceptions (4/13, 31%). The main variables affecting the levels of diagnostic and triage accuracy were the severity and urgency of the condition, the use of artificial intelligence algorithms, and demographic questions. However, the impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. All included studies had at least one area with unclear risk of bias according to the revised Quality Assessment of Diagnostic Accuracy Studies-2 tool.

CONCLUSIONS

Although OSCs have potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy before widescale adoption in community and health care settings. Future studies should aim to use a common methodology and agreed standard for evaluation to facilitate objective benchmarking and validation.

TRIAL REGISTRATION

PROSPERO CRD42020215210; https://tinyurl.com/3949zw83.

Engagement With the Centers for Disease Control and Prevention Coronavirus Self-Checker and Guidance Provided to Users in the United States From March 23, 2020, to April 19, 2021: Thematic and Trend Analysis

BACKGROUND

In 2020, at the onset of the COVID-19 pandemic, the United States experienced surges in healthcare needs, which challenged capacity throughout the healthcare system. Stay-at-home orders in many jurisdictions, cancellation of elective procedures, and closures of outpatient medical offices disrupted patient access to care. To inform symptomatic persons about when to seek care and potentially help alleviate the burden on the healthcare system, Centers for Disease Control and Prevention (CDC) and partners developed the CDC Coronavirus Self-Checker ("Self-Checker"). This interactive tool assists individuals seeking information about COVID-19 to determine the appropriate level of care by asking demographic, clinical, and nonclinical questions during an online "conversation."

OBJECTIVE

This paper describes user characteristics, trends in use, and recommendations delivered by the Self-Checker between March 23, 2020, and April 19, 2021, for pursuing appropriate levels of medical care depending on the severity of user symptoms.

METHODS

User characteristics and trends in completed conversations that resulted in a care message were analyzed. Care messages delivered by the Self-Checker were manually classified into three overarching conversation themes: (1) seek care immediately; (2) take no action, or stay home and self-monitor; and (3) conversation redirected. Trends in 7-day averages of conversations and COVID-19 cases were examined with development and marketing milestones that potentially impacted Self-Checker user engagement.

RESULTS

Among 16,718,667 completed conversations, the Self-Checker delivered recommendations for 69.27% (n=11,580,738) of all conversations to "take no action, or stay home and self-monitor"; 28.8% (n=4,822,138) of conversations to "seek care immediately"; and 1.89% (n=315,791) of conversations were redirected to other resources without providing any care advice. Among 6.8 million conversations initiated for self-reported sick individuals without life-threatening symptoms, 59.21% resulted in a recommendation to "take no action, or stay home and self-monitor." Nearly all individuals (99.8%) who were not sick were also advised to "take no action, or stay home and self-monitor."

CONCLUSIONS

The majority of Self-Checker conversations resulted in advice to take no action, or stay home and self-monitor. This guidance may have reduced patient volume on the medical system; however, future studies evaluating patients' satisfaction, intention to follow the care advice received, course of action, and care modality pursued could clarify the impact of the Self-Checker and similar tools during future public health emergencies.

A scoping review on the use and usefulness of online symptom checkers and triage systems: How to proceed?

Background

Patients are increasingly turning to the Internet for health information. Numerous online symptom checkers and digital triage tools are currently available to the general public in an effort to meet this need, simultaneously acting as a demand management strategy to aid the overburdened health care system. The implementation of these services requires an evidence-based approach, warranting a review of the available literature on this rapidly evolving topic.

Objective

This scoping review aims to provide an overview of the current state of the art and identify research gaps through an analysis of the strengths and weaknesses of the presently available literature.

Methods

A systematic search strategy was formed and applied to six databases: Cochrane library, NICE, DARE, NIHR, Pubmed, and Web of Science. Data extraction was performed by two researchers according to a pre-established data charting methodology allowing for a thematic analysis of the results.

Results

A total of 10,250 articles were identified, and 28 publications were found eligible for inclusion. Users of these tools are often younger, female, more highly educated and technologically literate, potentially impacting digital divide and health equity. Triage algorithms remain risk-averse, which causes challenges for their accuracy. Recent evolutions in algorithms have varying degrees of success. Results on impact are highly variable, with potential effects on demand, accessibility of care, health literacy and syndromic surveillance. Both patients and healthcare providers are generally positive about the technology and seem amenable to the advice given, but there are still improvements to be made toward a more patient-centered approach. The significant heterogeneity across studies and triage systems remains the primary challenge for the field, limiting transferability of findings.

Conclusion

Current evidence included in this review is characterized by significant variability in study design and outcomes, highlighting the significant challenges for future research.An evolution toward more homogeneous methodologies, studies tailored to the intended setting, regulation and standardization of evaluations, and a patient-centered approach could benefit the field.

Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Background

Although online self-triage is easily accessible, little is known about the patients who use self-triage or their subsequent diagnoses. We compared ear/hearing self-triage subsequent diagnoses to ear/hearing visit diagnoses in emergency departments (ED) and ambulatory clinics across the United States.

Methods

We compared International Classification of Diseases version 10 (ICD10) coded diagnoses following online self-triage for ear/hearing concerns with those from national ED and ambulatory clinic samples. We used data from the Centers for Disease Control (CDC) National Hospital Ambulatory Medical Care Survey (NHAMCS) and National Ambulatory Medical Care Survey (NAMCS) for comparison. Using matched ear/hearing diagnostic categories for those aged 1 and over, we compared self-triage diagnosis frequencies with national ED and ambulatory diagnosis frequencies.

Results

Following ear/hearing self-triage, there were 1092 subsequent office visits with a primary diagnosis code. For five frequently diagnosed ear/hearing conditions (i.e., suppurative and nonsuppurative otitis media [OM], otalgia, otitis externa, and cerumen impaction), there was a strong correlation between diagnosis counts made following self-triage and estimated counts of national ED visit diagnoses (r = 0.94; CI 95% [0.37 to 0.99]; p = .016, adjusted r2 = 0.85). Seven diagnoses were available to compare with the national ambulatory sample; correlation was r = 0.79; CI 95% [0.08 to 0.97]; p = .037, adjusted r2 = 0.54. For ages 1 and over, estimated hospital admissions from the national ED visits for ear/hearing were 0.76%, CI 95% [0.28-2.1%]; estimated total national ear/hearing ED visits were 7.5 million (for 4 years, 2016 through 2019).

Conclusion

The strong correlation of ear-related self-triage diagnoses with national ED diagnoses and the low hospitalization risk for these diagnoses suggests that there is an opportunity for self-triage of ear/hearing concerns to decrease ED visits for these symptoms.

Self-Triage Use, Subsequent Healthcare Utilization, and Diagnoses: A Retrospective Study of Process and Clinical Outcomes Following Self-Triage and Self-Scheduling for Ear or Hearing Symptoms

Background

Self-triage is becoming more widespread, but little is known about the people who are using online self-triage tools and their outcomes. For self-triage researchers, there are significant barriers to capturing subsequent healthcare outcomes. Our integrated healthcare system was able to capture subsequent healthcare utilization of individuals who used self-triage integrated with self-scheduling of provider visits.

Methods

We retrospectively examined healthcare utilization and diagnoses after patients had used self-triage and self-scheduling for ear or hearing symptoms. Outcomes and counts of office visits, telemedicine interactions, emergency department visits, and hospitalizations were captured. Diagnosis codes associated with subsequent provider visits were dichotomously categorized as being associated with ear or hearing concerns or not. Nonvisit care encounters of patient-initiated messages, nurse triage calls, and clinical communications were also captured.

Results

For 2168 self-triage uses, we were able to capture subsequent healthcare encounters within 7 days of the self-triage for 80.5% (1745/2168). In subsequent 1092 office visits with diagnoses, 83.1% (891/1092) of the uses were associated with relevant ear, nose and throat diagnoses. Only 0.24% (4/1662) of patients with captured outcomes were associated with a hospitalization within 7 days. Self-triage resulted in a self-scheduled office visit in 7.2% (126/1745). Office visits resulting from a self-scheduled visit had significantly fewer combined non-visit care encounters per office visit (fewer combined nurse triage calls, patient messages, and clinical communication messages) than office visits that were not self-scheduled (-0.51; 95% CI, -0.72 to -0.29; P < .0001).

Conclusion

In an appropriate healthcare setting, self-triage outcomes can be captured in a high percentage of uses to examine for safety, patient adherence to recommendations, and efficiency of self-triage. With the ear or hearing self-triage, most uses had subsequent visit diagnoses relevant to ear or hearing, so most patients appeared to be selecting the appropriate self-triage pathway for their symptoms.

Challenges in evaluating the accuracy of AI-containing digital triage systems: A systematic review

INTRODUCTION

Patient-operated digital triage systems with AI components are becoming increasingly common. However, previous reviews have found a limited amount of research on such systems' accuracy. This systematic review of the literature aimed to identify the main challenges in determining the accuracy of patient-operated digital AI-based triage systems.

METHODS

A systematic review was designed and conducted in accordance with PRISMA guidelines in October 2021 using PubMed, Scopus and Web of Science. Articles were included if they assessed the accuracy of a patient-operated digital triage system that had an AI-component and could triage a general primary care population. Limitations and other pertinent data were extracted, synthesized and analysed. Risk of bias was not analysed as this review studied the included articles' limitations (rather than results). Results were synthesized qualitatively using a thematic analysis.

RESULTS

The search generated 76 articles and following exclusion 8 articles (6 primary articles and 2 reviews) were included in the analysis. Articles' limitations were synthesized into three groups: epistemological, ontological and methodological limitations. Limitations varied with regards to intractability and the level to which they can be addressed through methodological choices. Certain methodological limitations related to testing triage systems using vignettes can be addressed through methodological adjustments, whereas epistemological and ontological limitations require that readers of such studies appraise the studies with limitations in mind.

DISCUSSION

The reviewed literature highlights recurring limitations and challenges in studying the accuracy of patient-operated digital triage systems with AI components. Some of these challenges can be addressed through methodology whereas others are intrinsic to the area of inquiry and involve unavoidable trade-offs. Future studies should take these limitations in consideration in order to better address the current knowledge gaps in the literature.

Self-care can be an alternative to expand access to universal health care: What policy makers, governments and implementers can consider for South Africa

As a result of collaboration amongst the various decision-makers in the field of healthcare, there has been an improvement in the access to healthcare and living conditions globally. Nonetheless, poorer communities continue to benefit the least from public investment. To bridge the gap, self-care can be a viable alternative as it allows individuals and communities to reduce their dependence on government healthcare solutions. Barriers to self-care do exist. Some of these are cost effectiveness, usability of self-care instruments, differentiated strategies and linkage to care. In identifying these obstacles, it is also worthwhile to investigate how they can be mitigated. To encourage sustained self-care in the HIV continuum, contextual factors as well as the manner in which individuals and communities engage with self-care must be considered. In South Africa, multiple variables come into play: literacy levels, cultural influences, socio-economic conditions and access to resources are some of these. Evidence demonstrates how self-care can be promoted by various stakeholders re-strategising to tweak and in some cases totally change existing structures. This paper explores some of the transformations, like at a governmental level where the supply of HIV self-testing kits is increased, at a production level where instructions for use are reformatting, in communities where sports programmes fulfil the dual purpose of developing sport skills and providing HIV education concurrently, and at an individual level where greater awareness invites greater participation in self-care. While self-care is a promising proposal, it is not a replacement for traditional health-care practices, but a complementary approach.

Ethical, legal, and social aspects of symptom checker applications: a scoping review

Symptom Checker Applications (SCA) are mobile applications often designed for the end-user to assist with symptom assessment and self-triage. SCA are meant to provide the user with easily accessible information about their own health conditions. However, SCA raise questions regarding ethical, legal, and social aspects (ELSA), for example, regarding fair access to this new technology. The aim of this scoping review is to identify the ELSA of SCA in the scientific literature. A scoping review was conducted to identify the ELSA of SCA. Ten databases (e.g., Web of Science and PubMed) were used. Studies on SCA that address ELSA, written in English or German, were included in the review. The ELSA of SCA were extracted and synthesized using qualitative content analysis. A total of 25,061 references were identified, of which 39 were included in the analysis. The identified aspects were allotted to three main categories: (1) Technology; (2) Individual Level; and (3) Healthcare system. The results show that there are controversial debates in the literature on the ethical and social challenges of SCA usage. Furthermore, the debates are characterised by a lack of a specific legal perspective and empirical data. The review provides an overview on the spectrum of ELSA regarding SCA. It offers guidance to stakeholders in the healthcare system, for example, patients, healthcare professionals, and insurance providers and could be used in future empirical research to investigate the perspectives of those affected, such as users.

WtsWrng Interim Comparative Effectiveness Evaluation and Description of the Challenges to Develop, Assess, and Introduce This Novel Digital Application in a Traditional Health System

Science and technology have evolved quickly during the two decades of the 21st century, but healthcare systems are grounded in last century's structure and processes. Changes in the way health care is provided are demanded; digital transformation is a key driver making healthcare systems more accessible, agile, efficient, and citizen-centered. Nevertheless, the way healthcare systems function challenges the development (Innovation + Development and regulatory requirements), assessment (methodological guidance weaknesses), and adoption of digital applications (DAs). WtsWrng (WW), an innovative DA which uses images to interact with citizens for symptom triage and monitoring, is used as an example to show the challenges faced in its development and clinical validation and how these are being overcome. To prove WW's value from inception, novel approaches for evidence generation that allows for an agile and patient-centered development have been applied. Early scientific advice from NICE (UK) was sought for study design, an iterative development and interim analysis was performed, and different statistical parameters (Kappa, B statistic) were explored to face development and assessment challenges. WW triage accuracy at cutoff time ranged from 0.62 to 0.94 for the most frequent symptoms attending the Emergency Department (ED), with the observed concordance for the 12 most frequent diagnostics at hospital discharge fluctuating between 0.4 to 0.97; 8 of the diagnostics had a concordance greater than 0.8. This experience should provoke reflective thinking for DA developers, digital health scientists, regulators, health technology assessors, and payers.

Patients' self-triage for unscheduled urgent care: a preliminary study on the accuracy and factors affecting the performance of a Belgian self-triage platform

Background

Management of unscheduled urgent care is a complex concern for many healthcare providers. Facing the challenge of appropriately dispatching unscheduled care, primary and emergency physicians have collaboratively implemented innovative strategies such as telephone triage. Currently, new original solutions tend to emerge with the development of new technologies. We created an interactive patient self-triage platform, ODISSEE, and aimed to explore its accuracy and potential factors affecting its performance using clinical case scenarios.

Methods

The ODISSEE platform was developed based on previously validated triage protocols for out-of-hours primary care. ODISSEE is composed of 18 icons leading to algorithmic questions that finally provide an advised orientation (emergency or primary care services). To investigate ODISSEE performance, we used 100 clinical case scenarios, each associated with a preestablished orientation determined by a group of experts. Fifteen volunteers were asked to self-triage with 50 randomly selected scenarios using ODISSEE on a digital tablet. Their triage results were compared with the experts’ references.

Results

The 15 participants performed a total of 750 self-triages, which matched the experts references regarding the level of care in 85.6% of the cases. The orientation was incorrect in 14.4%, with an undertriage rate of 1.9% and an overtriage rate of 12.5%. The tool’s specificity and sensitivity to advise participants on the appropriate level of care were 69% (95% CI: 64—74) and 97% (95% CI: 95—98) respectively. When combined with advice on the level of urgency, the tool only found the correct orientation in 68.4% with 9.2% of undertriages and 22.4% of overtriages. Some participant characteristics and the types of medical conditions demonstrated a significant association with the tool performance.

Conclusion

Self-triage apps, such as the ODISSEE platform, could represent an innovative method to allow patients to self-triage to the most appropriate level of care. This study based on clinical vignettes highlights some positive arguments regarding ODISSEE safety, but further research is needed to assess the generalizability of such tools to the population without equity issues.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08571-5.

User initiated symptom assessment with an electronic symptom checker. Study protocol for mixed-methods validation. (Preprint)

BACKGROUND

The national Omaolo digital social welfare and health care service of Finland provides a symptom checker, Omaolo, which is a medical device (based on Duodecim Clinical Decision Support EBMEDS software) with a CE marking (risk class IIa), manufactured by the government-owned DigiFinland Oy. Users of this service can perform their triage by using the questions in the symptom checker. By completing the symptom checker, the user receives a recommendation for action and a service assessment with appropriate guidance regarding their health problems on the basis of a selected specific symptom in the symptom checker. This allows users to be provided with appropriate health care services, regardless of time and place.

OBJECTIVE

This study describes the protocol for the mixed methods validation process of the symptom checker available in Omaolo digital services.

METHODS

This is a mixed methods study using quantitative and qualitative methods, which will be part of the clinical validation process that takes place in primary health care centers in Finland. Each organization provides a space where the study and the nurse triage can be done in order to include an unscreened target population of users. The primary health care units provide walk-in model services, where no prior phone call or contact is required. For the validation of the Omaolo symptom checker, case vignettes will be incorporated to supplement the triage accuracy of rare and acute cases that cannot be tested extensively in real-life settings. Vignettes are produced from a variety of clinical sources, and they test the symptom checker in different triage levels by using 1 standardized patient case example.

RESULTS

This study plan underwent an ethics review by the regional permission, which was requested from each organization participating in the research, and an ethics committee statement was requested and granted from Pirkanmaa hospital district's ethics committee, which is in accordance with the University of Tampere's regulations. Of 964 clinical user-filled symptom checker assessments, 877 cases were fully completed with a triage result, and therefore, they met the requirements for clinical validation studies. The goal for sufficient data has been reached for most of the chief symptoms. Data collection was completed in September 2019, and the first feasibility and patient experience results were published by the end of 2020. Case vignettes have been identified and are to be completed before further testing the symptom checker. The analysis and reporting are estimated to be finalized in 2024.

CONCLUSIONS

The primary goals of this multimethod electronic symptom checker study are to assess safety and to provide crucial information regarding the accuracy and usability of the Omaolo electronic symptom checker. To our knowledge, this will be the first study to include real-life clinical cases along with case vignettes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/41423.

Safety of Triage Self-assessment Using a Symptom Assessment App for Walk-in Patients in the Emergency Care Setting: Observational Prospective Cross-sectional Study

Background

Increasing use of emergency departments (EDs) by patients with low urgency, combined with limited availability of medical staff, results in extended waiting times and delayed care. Technological approaches could possibly increase efficiency by providing urgency advice and symptom assessments.

Objective

The purpose of this study is to evaluate the safety of urgency advice provided by a symptom assessment app, Ada, in an ED.

Methods

The study was conducted at the interdisciplinary ED of Marburg University Hospital, with data collection performed between August 2019 and March 2020. This study had a single-center cross-sectional prospective observational design and included 378 patients. The app’s urgency recommendation was compared with an established triage concept (Manchester Triage System [MTS]), including patients from the lower 3 MTS categories only. For all patients who were undertriaged, an expert physician panel assessed the case to detect potential avoidable hazardous situations (AHSs).

Results

Of 378 participants, 344 (91%) were triaged the same or more conservatively and 34 (8.9%) were undertriaged by the app. Of the 378 patients, 14 (3.7%) had received safe advice determined by the expert panel and 20 (5.3%) were considered to be potential AHS. Therefore, the assessment could be considered safe in 94.7% (358/378) of the patients when compared with the MTS assessment. From the 3 lowest MTS categories, 43.4% (164/378) of patients were not considered as emergency cases by the app, but could have been safely treated by a general practitioner or would not have required a physician consultation at all.

Conclusions

The app provided urgency advice after patient self-triage that has a high rate of safety, a rate of undertriage, and a rate of triage with potential to be an AHS, equivalent to telephone triage by health care professionals while still being more conservative than direct ED triage. A large proportion of patients in the ED were not considered as emergency cases, which could possibly relieve ED burden if used at home. Further research should be conducted in the at-home setting to evaluate this hypothesis.

Trial Registration

German Clinical Trial Registration DRKS00024909; https://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00024909

Improving Emergency Department Patient-Physician Conversation Through an Artificial Intelligence Symptom-Taking Tool: Mixed Methods Pilot Observational Study

BACKGROUND

Establishing rapport and empathy between patients and their health care provider is important but challenging in the context of a busy and crowded emergency department (ED).

OBJECTIVE

We explore the hypotheses that rapport building, documentation, and time efficiency might be improved in the ED by providing patients a digital tool that uses Bayesian reasoning-based techniques to gather relevant symptoms and history for handover to clinicians.

METHODS

A 2-phase pilot evaluation was carried out in the ED of a German tertiary referral and major trauma hospital that treats an average of 120 patients daily. Phase 1 observations guided iterative improvement of the digital tool, which was then further evaluated in phase 2. All patients who were willing and able to provide consent were invited to participate, excluding those with severe injury or illness requiring immediate treatment, with traumatic injury, incapable of completing a health assessment, and aged <18 years. Over an 18-day period with 1699 patients presenting to the ED, 815 (47.96%) were eligible based on triage level. With available recruitment staff, 135 were approached, of whom 81 (60%) were included in the study. In a mixed methods evaluation, patients entered information into the tool, accessed by clinicians through a dashboard. All users completed evaluation Likert-scale questionnaires rating the tool's performance. The feasibility of a larger trial was evaluated through rates of recruitment and questionnaire completion.

RESULTS

Respondents strongly endorsed the tool for facilitating conversation (61/81, 75% of patients, 57/78, 73% of physician ratings, and 10/10, 100% of nurse ratings). Most nurses judged the tool as potentially time saving, whereas most physicians only agreed for a subset of medical specialties (eg, surgery). Patients reported high usability and understood the tool's questions. The tool was recommended by most patients (63/81, 78%), in 53% (41/77) of physician ratings, and in 76% (61/80) of nurse ratings. Questionnaire completion rates were 100% (81/81) by patients and 96% (78/81 enrolled patients) by physicians.

CONCLUSIONS

This pilot confirmed that a larger study in the setting would be feasible. The tool has clear potential to improve patient-health care provider interaction and could also contribute to ED efficiency savings. Future research and development will extend the range of patients for whom the history-taking tool has clinical utility.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00024115; https://drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00024115.

Use of the McIsaac Score to Predict Group A Streptococcal Pharyngitis in Outpatient Nurse Phone Triage and Electronic Visits Compared With In-Person Visits: Retrospective Observational Study

Background

The McIsaac criteria are a validated scoring system used to determine the likelihood of an acute sore throat being caused by group A streptococcus (GAS) to stratify patients who need strep testing.

Objective

We aim to compare McIsaac criteria obtained during face-to-face (f2f) and non-f2f encounters.

Methods

This retrospective study compared the percentage of positive GAS tests by McIsaac score for scores calculated during nurse protocol phone encounters, e-visits (electronic visits), and in person f2f clinic visits.

Results

There was no difference in percentages of positive strep tests between encounter types for any of the McIsaac scores. There were significantly more phone and e-visit encounters with any missing score components compared with f2f visits. For individual score components, there were significantly fewer e-visits missing fever and cough information compared with phone encounters and f2f encounters. F2f encounters were significantly less likely to be missing descriptions of tonsils and lymphadenopathy compared with phone and e-visit encounters. McIsaac scores of 4 had positive GAS rates of 55% to 68% across encounter types. There were 4 encounters not missing any score components with a McIsaac score of 0. None of these 4 encounters had a positive GAS test.

Conclusions

McIsaac scores of 4 collected during non-f2f care could be used to consider empiric treatment for GAS without testing if significant barriers to testing exist such as the COVID-19 pandemic or geographic barriers. Future studies should evaluate further whether non-f2f encounters with McIsaac scores of 0 can be safely excluded from GAS testing.

Validation of a Musculoskeletal Digital Assessment Routing Tool: Protocol for a Pilot Randomized Crossover Noninferiority Trial

Background

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing demand for health care use. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are rarely tested with sufficient rigor in clinical trials, which is the gold standard for clinical proof of safety and efficacy. We have developed a new musculoskeletal digital assessment routing tool (DART) that allows users to self-assess and be directed to the right care. DART requires validation in a real-world setting before implementation.

Objective

This pilot study aims to assess the feasibility of a future trial by exploring the key aspects of trial methodology, assessing the procedures, and collecting exploratory data to inform the design of a definitive randomized crossover noninferiority trial to assess DART safety and effectiveness.

Methods

We will collect data from 76 adults with a musculoskeletal condition presenting to general practitioners within a National Health Service (NHS) in England. Participants will complete both a DART assessment and a physiotherapist-led triage, with the order determined by randomization. The primary analysis will involve an absolute agreement intraclass correlation (A,1) estimate with 95% CI between DART and the clinician for assessment outcomes signposting to condition management pathways. Data will be collected to allow the analysis of participant recruitment and retention, randomization, allocation concealment, blinding, data collection process, and bias. In addition, the impact of trial burden and potential barriers to intervention delivery will be considered. The DART user satisfaction will be measured using the system usability scale.

Results

A UK NHS ethics submission was done during June 2021 and is pending approval; recruitment will commence in early 2022, with data collection anticipated to last for 3 months. The results will be reported in a follow-up paper in 2022.

Conclusions

This study will inform the design of a randomized controlled crossover noninferiority study that will provide evidence concerning mobile health DART system clinical signposting in an NHS setting before real-world implementation. Success should produce evidence of a safe, effective system with good usability, potentially facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This rigorous approach to mobile health system testing could be used as a guide for other developers of similar applications.

Trial Registration

ClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029

International Registered Report Identifier (IRRID)

PRR1-10.2196/31541

Performance of a new symptom checker in patient triage: Canadian cohort study

BACKGROUND

Computerized algorithms known as symptom checkers aim to help patients decide what to do should they have a new medical concern. However, despite widespread implementation, most studies on symptom checkers have involved simulated patients. Only limited evidence currently exists about symptom checker safety or accuracy when used by real patients. We developed a new prototype symptom checker and assessed its safety and accuracy in a prospective cohort of patients presenting to primary care and emergency departments with new medical concerns.

METHOD

A prospective cohort study was done to assess the prototype's performance. The cohort consisted of adult patients (≥16 years old) who presented to hospital emergency departments and family physician clinics. Primary outcomes were safety and accuracy of triage recommendations to seek hospital care, seek primary care, or manage symptoms at home.

RESULTS

Data from 281 hospital patients and 300 clinic patients were collected and analyzed. Sensitivity to emergencies was 100% (10/10 encounters). Sensitivity to urgencies was 90% (73/81) and 97% (34/35) for hospital and primary care patients, respectively. The prototype was significantly more accurate than patients at triage (73% versus 58%, p<0.01). Compliance with triage recommendations in this cohort using this iteration of the symptom checker would have reduced hospital visits by 55% but cause potential harm in 2-3% from delay in care.

INTERPRETATION

The prototype symptom checker was superior to patients in deciding the most appropriate treatment setting for medical issues. This symptom checker could reduce a significant number of unnecessary hospital visits, with accuracy and safety outcomes comparable to existing data on telephone triage.

Trust Me, I’m Not a Doctor! Determinants of Laypersons’ Trust in Medical Decision Aids: Experimental Study (Preprint)

Background

Objective

Methods

Results

Conclusions

Trial Registration

Prevalence, aetiologies and prognosis of the symptom cough in primary care: a systematic review and meta-analysis

Background

Cough is a relevant reason for encounter in primary care. For evidence-based decision making, general practitioners need setting-specific knowledge about prevalences, pre-test probabilities, and prognosis. Accordingly, we performed a systematic review of symptom-evaluating studies evaluating cough as reason for encounter in primary care.

Methods

We conducted a search in MEDLINE and EMBASE. Eligibility criteria and methodological quality were assessed independently by two reviewers. We extracted data on prevalence, aetiologies and prognosis, and estimated the variation across studies. If justifiable in terms of heterogeneity, we performed a meta-analysis.

Results

We identified 21 eligible studies on prevalence, 12 on aetiology, and four on prognosis. Prevalence/incidence estimates were 3.8–4.2%/12.5% (Western primary care) and 10.3–13.8%/6.3–6.5% in Africa, Asia and South America. In Western countries the underlying diagnoses for acute cough or cough of all durations were respiratory tract infections (73–91.9%), influenza (6–15.2%), asthma (3.2–15%), laryngitis/tracheitis (3.6–9%), pneumonia (4.0–4.2%), COPD (0.5–3.3%), heart failure (0.3%), and suspected malignancy (0.2–1.8%). Median time for recovery was 9 to 11 days. Complete recovery was reported by 40.2- 67% of patients after two weeks, and by 79% after four weeks. About 21.1–35% of patients re-consulted; 0–1.3% of acute cough patients were hospitalized, none died. Evidence is missing concerning subacute and chronic cough.

Conclusion

Prevalences and incidences of cough are high and show regional variation. Acute cough, mainly caused by respiratory tract infections, is usually self-limiting (supporting a “wait-and-see” strategy). We have no setting-specific evidence to support current guideline recommendations concerning subacute or chronic cough in Western primary care. Our study presents epidemiological data under non non-pandemic conditions. It will be interesting to compare these data to future research results of the post-pandemic era.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12875-021-01501-0.

Emergency department crowding: why do patients walk-in?

Objectives: For years, general practitioners (GP) shortage and patients' increasing demand for acute care have been associated with Emergency Department (ED) crowding. Indeed, EDs admissions for non-emergency care seem to constantly increase. Surprisingly, the rationale for patients own decision to directly reach EDs over primary care have been poorly investigated to date.Methods: We conducted a study on patients admitted in two University EDs during nine consecutive days. Patients were asked to answer a survey about their frames for coming and if they were self-referred, referred by a GP, a specialist or after calling the Emergency Number.Results: During the study period, 68.0% of patients were self-referred, 17.0% referred by their GP, 8.5% by a specialist and 7% after an emergency call. 51.0% of the self-referrals thought EDs were the appropriate location to deal with their health problem and 24.0% because of a better accessibility. We noticed that 15.0% of the incomings looked for specialized care and 4.22% reported that the stress had motivated them. Of note, 4.6% of the patients were attracted by the hospital reputation. Financial concerns represented less than 1.0% of the motives invocated.Conclusion: We found that patients' self-perceived severity of illness is the predominant frame to each the ED when they face needs for acute care. EDs' accessibility as compared with other facilities also seems to encourage patients to come to the ED. Other factors such as the hospital reputation or patients' stress tend to influence ED attendance but to a much lesser extent.

Experience and perspectives of users and non-users of the Ask your pharmacist teleconsultation platform

Background

There is a growing trend concerning the use of information and communication technologies (ICT) for seeking health-related information such as information on medications and side effects. However, people looking online for health information cannot always judge the credibility of the information.

Objective(s)

This study aimed to describe patients' and pharmacists' experience using an asynchronous teleconsultation platform entitled “Ask Your Pharmacist” (AYP) and gather their perspectives and those of various healthcare and social services professionals providing primary care.

Methods

We performed semi-structured individual interviews over the telephone with patients having used the platform, pharmacists providing teleconsultation services on the platform, and various professionals delivering healthcare and psychosocial services to ambulatory patients. The questions explored specific themes, such as the perceived utility and impacts of the platform. We transcribed the interviews and performed a content analysis.

Results

We interviewed eight patients, six AYP pharmacists, and 15 healthcare and social services professionals. Participants perceived that the platform was simple to use and accessible. They also perceived that AYP promoted the visibility and the value of pharmacists' services. Some constraints were also shared, notably regarding questions requiring immediate attention or about complex situations.

Conclusions

The experience and perspectives of users and non-users of the AYP platform are mostly positive, but concerns were also raised regarding patients' safety. Results suggest that AYP could be a complementary tool to offer to ambulatory patients for simple, general and non-urgent problems.

Patient triage system for supporting the operation of dispatch centres and rescue teams

BACKGROUND

The Thai medical application for patient triage, namely Triagist, is an mHealth application designed to support the pre-hospital process. However, since the functions of the application that are necessary for the pre-hospital process have been found not to be fully developed, the addition of a back-end system has been considered to increase its performance and usability.

OBJECTIVE

To determine the ability of the previous version to effectively manage the pre-hospital process and analyse the current problems with the pre-hospital operation. Therefore, the new system was developed to support the connection of dispatch centres or operational centres to the Triagist mobile application and system evaluation.

METHOD

Design thinking methodology was used to analyse, design and develop a patient triage system to support the pre-hospital process in Thailand based on users' requirements. 68 active members of the rescue teams and emergency medical staff in Chiang Mai and Lampang provinces were recruited to test the reliability of the system based on a prototype application.

RESULTS

The new medical mobile application for patient triage in Thailand was validated for use due to containing the two essential functions of Initial Dispatch Code (IDC) geolocation and IDC management. When the system was tested by emergency staff who were responsible for using it, those with the least experience were found to use it better than their highly experienced colleagues. Moreover, in cases where the system had been implemented, it was found to determine the frequency of symptoms, the time period during which cases occurred, and the density of cases in each area.

CONCLUSION

This system, which has been developed based on the use of smart technology, will play an important role in supporting emergency services in Thailand by enhancing the efficiency of the pre-hospital process. Emergency centres will receive IDC information from the geolocation system so that they can determine patients' location without undue delay. Emergency services will be able to rapidly prepare the necessary resources and administrative tasks will be supported by linking the dispatch centre to central rescue teams.

Implementation of a Self-Triage Web Application for Suspected COVID-19 and Its Impact on Emergency Call Centers: Observational Study

Background

We developed a self-triage web application for COVID-19 symptoms, which was launched in France in March 2020, when French health authorities recommended all patients with suspected COVID-19 call an emergency phone number.

Objective

Our objective was to determine if a self-triage tool could reduce the burden on emergency call centers and help predict increasing burden on hospitals.

Methods

Users were asked questions about their underlying conditions, sociodemographic status, postal code, and main COVID-19 symptoms. Participants were advised to call an emergency call center if they reported dyspnea or complete loss of appetite for over 24 hours. Data on COVID-19–related calls were collected from 6 emergency call centers and data on COVID-19 hospitalizations were collected from Santé Publique France and the French Ministry of Health. We examined the change in the number of emergency calls before and after the launch of the web application.

Results

From March 17 to April 2, 2020, 735,419 questionnaires were registered in the study area. Of these, 121,370 (16.5%) led to a recommendation to call an emergency center. The peak number of overall questionnaires and of questionnaires leading to a recommendation to call an emergency center were observed on March 22, 2020. In the 17 days preceding the launch of the web application, emergency call centers in the study area registered 66,925 COVID-19–related calls and local hospitals admitted 639 patients for COVID-19; the ratio of emergency calls to hospitalizations for COVID-19 was 104.7 to 1. In the 17 days following the launch of the web application, there were 82,347 emergency calls and 6009 new hospitalizations for COVID-19, a ratio of 13.7 calls to 1 hospitalization (chi-square test: P<.001).

Conclusions

The self-triage web application launch was followed by a nearly 10-fold increase in COVID-19–related hospitalizations with only a 23% increase in emergency calls. The peak of questionnaire completions preceded the peak of COVID-19–related hospitalizations by 5 days. Although the design of this study does not allow us to conclude that the self-triage tool alone contributed to the alleviation of calls to the emergency call centers, it does suggest that it played a role, and may be used for predicting increasing burden on hospitals.

Trial Registration

ClinicalTrials.gov NCT04331171; https://clinicaltrials.gov/ct2/show/NCT04331171

[Artificial intelligence and smartphone program applications (Apps) : Relevance for dermatological practice]

ADVANTAGES OF ARTIFICIAL INTELLIGENCE (AI)

With responsible, safe and successful use of artificial intelligence (AI), possible advantages in the field of dermato-oncology include the following: (1) medical work can focus on skin cancer patients, (2) patients can be more quickly and effectively treated despite the increasing incidence of skin cancer and the decreasing number of actively working dermatologists and (3) users can learn from the AI results. POTENTIAL DISADVANTAGES AND RISKS OF AI USE: (1) Lack of mutual trust can develop due to the decreased patient-physician contact, (2) additional time effort will be necessary to promptly evaluate the AI-classified benign lesions, (3) lack of adequate medical experience to recognize misclassified AI decisions and (4) recontacting a patient in due time in the case of incorrect AI classifications. Still problematic in the use of AI are the medicolegal situation and remuneration. Apps using AI currently cannot provide sufficient assistance based on clinical images of skin cancer.

REQUIREMENTS AND POSSIBLE USE OF SMARTPHONE PROGRAM APPLICATIONS

Smartphone program applications (apps) can be implemented responsibly when the image quality is good, the patient's history can be entered easily, transmission of the image and results are assured and medicolegal aspects as well as remuneration are clarified. Apps can be used for disease-specific information material and can optimize patient care by using teledermatology.

Self-referred walk-in patients in the emergency department - who and why? Consultation determinants in a multicenter study of respiratory patients in Berlin, Germany

BACKGROUND

Emergency department (ED) consultations are on the rise, and frequently consultations by non-urgent patients have been held accountable. Self-referred walk-in (SRW) consulters supposedly represent a predominantly less urgent patient population. The EMACROSS study aimed to explore consultation determinants and motives in SRW patients with respiratory symptoms.

METHODS

Multicenter survey of adult ED patients with respiratory complaints in eight emergency departments in central Berlin, Germany. Secondary hospital records data including diagnoses was additionally assessed. Characteristics of SRW and non-SRW patients were compared. Determinants of SRW consultation were evaluated by binary logistic regression. Consultation motives were analyzed descriptively. As a supplemental approach, network analysis (lasso-regularized mixed graphical model) was performed to explore connections between consultation determinants, consultation features and motives.

RESULTS

Between June 2017 and November 2018, n = 472 participants were included, the median age was 55 years (range 18-96), 53.2% of patients were male and n = 185 cases (39.2%) were SRW consulters. The SRW group showed lower proportions of potentially severe (pneumonia and respiratory failure, p < 0.001, χ2 test) and chronic pulmonary conditions. Determinants of SRW consultation identified by logistic regression were younger age (p < 0.001), tertiary education (p = 0.032), being a first-generation migrant (p = 0.002) or tourist (p = 0.008), having no regular primary care provider (p = 0.036) and no chronic pulmonary illness (p = 0.017). The area under the curve (AUC) for the model was 0.79. Personal distress and access problems in ambulatory care were stated most frequently as consultation motives in the SRW group; network analysis showed the scarcity of associations between demographic and medical SRW determinants and motives triggering the actual decision to consult.

CONCLUSIONS

As to "who" consults, this study identified demographic and medical predictors of SRW utilization. The said markers seem only remotely connected to "why" people decide for SRW visits. To alleviate ED crowding by addressing frequent SRW consultation motives, interventions focused on the ability for symptom self-assessment and at better-accessible alternative care seem sensible.

TRIAL REGISTRATION

German Clinical Trials Register ( DRKS00011930 ); date: 2017/04/25.

Human- Versus Machine Learning-Based Triage Using Digitalized Patient Histories in Primary Care: Comparative Study

BACKGROUND

Smartphones have made it possible for patients to digitally report symptoms before physical primary care visits. Using machine learning (ML), these data offer an opportunity to support decisions about the appropriate level of care (triage).

OBJECTIVE

The purpose of this study was to explore the interrater reliability between human physicians and an automated ML-based triage method.

METHODS

After testing several models, a naïve Bayes triage model was created using data from digital medical histories, capable of classifying digital medical history reports as either in need of urgent physical examination or not in need of urgent physical examination. The model was tested on 300 digital medical history reports and classification was compared with the majority vote of an expert panel of 5 primary care physicians (PCPs). Reliability between raters was measured using both Cohen κ (adjusted for chance agreement) and percentage agreement (not adjusted for chance agreement).

RESULTS

Interrater reliability as measured by Cohen κ was 0.17 when comparing the majority vote of the reference group with the model. Agreement was 74% (138/186) for cases judged not in need of urgent physical examination and 42% (38/90) for cases judged to be in need of urgent physical examination. No specific features linked to the model's triage decision could be identified. Between physicians within the panel, Cohen κ was 0.2. Intrarater reliability when 1 physician retriaged 50 reports resulted in Cohen κ of 0.55.

CONCLUSIONS

Low interrater and intrarater agreement in triage decisions among PCPs limits the possibility to use human decisions as a reference for ML to automate triage in primary care.

Rapid design and implementation of an integrated patient self-triage and self-scheduling tool for COVID-19

OBJECTIVE

To rapidly deploy a digital patient-facing self-triage and self-scheduling tool in a large academic health system to address the COVID-19 pandemic.

MATERIALS AND METHODS

We created a patient portal-based COVID-19 self-triage and self-scheduling tool and made it available to all primary care patients at the University of California, San Francisco Health, a large academic health system. Asymptomatic patients were asked about exposure history and were then provided relevant information. Symptomatic patients were triaged into 1 of 4 categories-emergent, urgent, nonurgent, or self-care-and then connected with the appropriate level of care via direct scheduling or telephone hotline.

RESULTS

This self-triage and self-scheduling tool was designed and implemented in under 2 weeks. During the first 16 days of use, it was completed 1129 times by 950 unique patients. Of completed sessions, 315 (28%) were by asymptomatic patients, and 814 (72%) were by symptomatic patients. Symptomatic patient triage dispositions were as follows: 193 emergent (24%), 193 urgent (24%), 99 nonurgent (12%), 329 self-care (40%). Sensitivity for detecting emergency-level care was 87.5% (95% CI 61.7-98.5%).

DISCUSSION

This self-triage and self-scheduling tool has been widely used by patients and is being rapidly expanded to other populations and health systems. The tool has recommended emergency-level care with high sensitivity, and decreased triage time for patients with less severe illness. The data suggests it also prevents unnecessary triage messages, phone calls, and in-person visits.

CONCLUSION

Patient self-triage tools integrated into electronic health record systems have the potential to greatly improve triage efficiency and prevent unnecessary visits during the COVID-19 pandemic.

Digital interventions for parents of acutely ill children and their treatment-seeking behaviour: a systematic review

BACKGROUND

Consultations for self-limiting infections in children are increasing. It has been proposed that digital technology could be used to enable parents' decision making in terms of self-care and treatment seeking.

AIM

To evaluate the evidence that digital interventions facilitate parents deciding whether to self-care or seek treatment for acute illnesses in children.

DESIGN AND SETTING

Systematic review of studies undertaken worldwide.

METHOD

Searches of MEDLINE and EMBASE were made to identify studies (of any design) published between database inception and January 2019 that assessed digital interventions for parents of children (from any healthcare setting) with acute illnesses. The primary outcome of interest was whether the use of digital interventions reduced the use of urgent care services.

RESULTS

Three studies were included in the review. They assessed two apps and one website: Children's On-Call - a US advice-only app; Should I See a Doctor? - a Dutch self-triage app for any acute illness; and Strategy for Off-Site Rapid Triage (SORT) for Kids - a US self-triage website for influenza-like illness. None of the studies involved parents during intervention development and it was shown that many parents did not find the two apps easy to use. The sensitivity of self-triage interventions was 84% for Should I See a Doctor? compared with nurse triage, and 93.3% for SORT for Kids compared with the need for emergency-department intervention; however, both had lower specificity (74% and 13%, respectively). None of the interventions demonstrated reduced use of urgent-care services.

CONCLUSION

There is little evidence to support the use of digital interventions to help parent and/or carers looking after children with acute illness. Future research should involve parents during intervention development, and adequately powered trials are needed to assess the impact of such interventions on health services and the identification of children who are seriously ill.

The Use of Smartphone-Based Triage to Reduce the Rate of Outpatient Error Registration: Cross-Sectional Study

BACKGROUND

In many clinics, patients now have the option to make Web-based appointments but doing so according to their own judgment may lead to wrong registration and delayed medical services. We hypothesized that smartphone-based triage in outpatient services is superior to Web-based self-appointment registration guided by the medical staff.

OBJECTIVE

This study aimed to investigate smartphone-based triage in outpatient services compared with Web-based self-appointment registration and to provide a reference for improving outpatient care under appointment registration.

METHODS

The following parameters in Guangzhou Women and Children's Medical Center were analyzed: wrong registration rate, the degree of patient satisfaction, outpatient visits 6 months before and after smartphone-based triage, queries after smartphone-based triage, number of successful registrations, inquiry content, and top 10 recommended diseases and top 10 recommended departments after queries.

RESULTS

Smartphone-based triage showed significant effects on average daily queries, which accounted for 16.15% (1956/12,112) to 29.46% (3643/12,366) of daily outpatient visits. The average daily successful registration after queries accounted for 56.14% (1101/1961) to 60.92% (1437/2359) of daily queries and 9.33% (1130/12,112) to 16.83% (2081/12,366) of daily outpatient visits. The wrong registration rate after smartphone-based triage was reduced from 0.68% (12,810/1,895,829) to 0.12% (2379/2,017,921) (P<.001), and the degree of patient satisfaction was improved. Monthly outpatient visits were increased by 0.98% (3192/325,710) to 13.09% (42,939/328,032) compared with the same period the preceding year (P=.02).

CONCLUSIONS

Smartphone-based triage significantly reduces the wrong registration rate caused by patient Web-based appointment registration and improves the degree of patient satisfaction. Thus, it is worth promoting.

Accuracy of a Chatbot (Ada) in the Diagnosis of Mental Disorders: Comparative Case Study With Lay and Expert Users

Background

Health apps for the screening and diagnosis of mental disorders have emerged in recent years on various levels (eg, patients, practitioners, and public health system). However, the diagnostic quality of these apps has not been (sufficiently) tested so far.

Objective

The objective of this pilot study was to investigate the diagnostic quality of a health app for a broad spectrum of mental disorders and its dependency on expert knowledge.

Methods

Two psychotherapists, two psychology students, and two laypersons each read 20 case vignettes with a broad spectrum of mental disorders. They used a health app (Ada—Your Health Guide) to get a diagnosis by entering the symptoms. Interrater reliabilities were computed between the diagnoses of the case vignettes and the results of the app for each user group.

Results

Overall, there was a moderate diagnostic agreement (kappa=0.64) between the results of the app and the case vignettes for mental disorders in adulthood and a low diagnostic agreement (kappa=0.40) for mental disorders in childhood and adolescence. When psychotherapists applied the app, there was a good diagnostic agreement (kappa=0.78) regarding mental disorders in adulthood. The diagnostic agreement was moderate (kappa=0.55/0.60) for students and laypersons. For mental disorders in childhood and adolescence, a moderate diagnostic quality was found when psychotherapists (kappa=0.53) and students (kappa=0.41) used the app, whereas the quality was low for laypersons (kappa=0.29). On average, the app required 34 questions to be answered and 7 min to complete.

Conclusions

The health app investigated here can represent an efficient diagnostic screening or help function for mental disorders in adulthood and has the potential to support especially diagnosticians in their work in various ways. The results of this pilot study provide a first indication that the diagnostic accuracy is user dependent and improvements in the app are needed especially for mental disorders in childhood and adolescence.

Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review

OBJECTIVES

In England, the NHS111 service provides assessment and triage by telephone for urgent health problems. A digital version of this service has recently been introduced. We aimed to systematically review the evidence on digital and online symptom checkers and similar services.

DESIGN

Systematic review.

DATA SOURCES

We searched Medline, Embase, the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Health Management Information Consortium, Web of Science and ACM Digital Library up to April 2018, supplemented by phrase searches for known symptom checkers and citation searching of key studies.

ELIGIBILITY CRITERIA

Studies of any design that evaluated a digital or online symptom checker or health assessment service for people seeking advice about an urgent health problem.

DATA EXTRACTION AND SYNTHESIS

Data extraction and quality assessment (using the Cochrane Collaboration version of QUADAS for diagnostic accuracy studies and the National Heart, Lung and Blood Institute tool for observational studies) were done by one reviewer with a sample checked for accuracy and consistency. We performed a narrative synthesis of the included studies structured around pre-defined research questions and key outcomes.

RESULTS

We included 29 publications (27 studies). Evidence on patient safety was weak. Diagnostic accuracy varied between different systems but was generally low. Algorithm-based triage tended to be more risk averse than that of health professionals. There was very limited evidence on patients' compliance with online triage advice. Study participants generally expressed high levels of satisfaction, although in mainly uncontrolled studies. Younger and more highly educated people were more likely to use these services.

CONCLUSIONS

The English 'digital 111' service has been implemented against a background of uncertainty around the likely impact on important outcomes. The health system may need to respond to short-term changes and/or shifts in demand. The popularity of online and digital services with younger and more educated people has implications for health equity.

PROSPERO REGISTRATION NUMBER

CRD42018093564.