Are e-health web users looking for different symptom information than callers to triage centers?

Service use, clinical outcomes and user experience associated with urgent care services that use telephone-based digital triage: a systematic review

OBJECTIVE

To evaluate service use, clinical outcomes and user experience related to telephone-based digital triage in urgent care.

DESIGN

Systematic review and narrative synthesis.

DATA SOURCES

Medline, Embase, CINAHL, Web of Science and Scopus were searched for literature published between 1 March 2000 and 1 April 2020.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Studies of any design investigating patterns of triage advice, wider service use, clinical outcomes and user experience relating to telephone based digital triage in urgent care.

DATA EXTRACTION AND SYNTHESIS

Two reviewers extracted data and conducted quality assessments using the mixed methods appraisal tool. Narrative synthesis was used to analyse findings.

RESULTS

Thirty-one studies were included, with the majority being UK based; most investigated nurse-led digital triage (n=26). Eight evaluated the impact on wider healthcare service use following digital triage implementation, typically reporting reduction or no change in service use. Six investigated patient level service use, showing mixed findings relating to patients' adherence with triage advice. Evaluation of clinical outcomes was limited. Four studies reported on hospitalisation rates of digitally triaged patients and highlighted potential triage errors where patients appeared to have not been given sufficiently high urgency advice. Overall, service users reported high levels of satisfaction, in studies of both clinician and non-clinician led digital triage, but with some dissatisfaction over the relevance and number of triage questions.

CONCLUSIONS

Further research is needed into patient level service use, including patients' adherence with triage advice and how this influences subsequent use of services. Further evaluation of clinical outcomes using larger datasets and comparison of different digital triage systems is needed to explore consistency and safety. The safety and effectiveness of non-clinician led digital triage also needs evaluation. Such evidence should contribute to improvement of digital triage tools and service delivery.

PROSPERO REGISTRATION NUMBER

CRD42020178500.

An evaluation of service user experience, clinical outcomes and service use associated with urgent care services that utilise telephone-based digital triage: a systematic review protocol

BACKGROUND

Telephone-based digital triage is widely used by services that provide urgent care. This involves a call handler or clinician using a digital triage tool to generate algorithm-based care advice, based on a patient's symptoms. Advice typically takes the form of signposting within defined levels of urgency to specific services or self-care advice. Despite wide adoption, there is limited evaluation of its impact on service user experience, service use and clinical outcomes; no previous systematic reviews have focussed on services that utilise digital triage, and its impact on these outcome areas within urgent care. This review aims to address this need, particularly now that telephone-based digital triage is well established in healthcare delivery.

METHODS

Studies assessing the impact of telephone-based digital triage on service user experience, health care service use and clinical outcomes will be identified through searches conducted in Medline, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science and Scopus. Search terms using words relating to digital triage and urgent care settings (excluding in-hours general practice) will be used. The review will include all original study types including qualitative, quantitative and mixed methods studies; studies published in the last 20 years and studies published in English. Quality assessment of studies will be conducted using the Mixed Methods Appraisal Tool (MMAT); a narrative synthesis approach will be used to analyse and summarise findings.

DISCUSSION

This is the first systematic review to evaluate service user experience, service use and clinical outcomes related to the use of telephone-based digital triage in urgent care settings. It will evaluate evidence from studies of wide-ranging designs. The narrative synthesis approach will enable the integration of findings to provide new insights on service delivery. Models of urgent care continue to evolve rapidly, with the emergence of self-triage tools and national help lines. Findings from this review will be presented in a practical format that can feed into the design of digital triage tools, future service design and healthcare policy.

SYSTEMATIC REVIEW REGISTRATION

This systematic review is registered on the international database of prospectively registered systematic reviews in health and social care (PROSPERO 2020 CRD42020178500 ).

Characterizing online searches in pediatric dermatology

The use of online medical information is a common trend but little is known about these searches and implications in pediatric dermatology. Through a convenience sample of 95 parents/guardians and patients in pediatric dermatology at the University of Minnesota, 38.9% of participants reported that they had used an online search tool prior to their appointment. Google was the most common tool used (91.4%), and eczematous conditions were the most common patient-suspected diagnosis (32.4%). Of interest, 13.9% of individuals became more concerned and 16.7% made an appointment earlier because of their online findings.

Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review

OBJECTIVES

In England, the NHS111 service provides assessment and triage by telephone for urgent health problems. A digital version of this service has recently been introduced. We aimed to systematically review the evidence on digital and online symptom checkers and similar services.

DESIGN

Systematic review.

DATA SOURCES

We searched Medline, Embase, the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Health Management Information Consortium, Web of Science and ACM Digital Library up to April 2018, supplemented by phrase searches for known symptom checkers and citation searching of key studies.

ELIGIBILITY CRITERIA

Studies of any design that evaluated a digital or online symptom checker or health assessment service for people seeking advice about an urgent health problem.

DATA EXTRACTION AND SYNTHESIS

Data extraction and quality assessment (using the Cochrane Collaboration version of QUADAS for diagnostic accuracy studies and the National Heart, Lung and Blood Institute tool for observational studies) were done by one reviewer with a sample checked for accuracy and consistency. We performed a narrative synthesis of the included studies structured around pre-defined research questions and key outcomes.

RESULTS

We included 29 publications (27 studies). Evidence on patient safety was weak. Diagnostic accuracy varied between different systems but was generally low. Algorithm-based triage tended to be more risk averse than that of health professionals. There was very limited evidence on patients' compliance with online triage advice. Study participants generally expressed high levels of satisfaction, although in mainly uncontrolled studies. Younger and more highly educated people were more likely to use these services.

CONCLUSIONS

The English 'digital 111' service has been implemented against a background of uncertainty around the likely impact on important outcomes. The health system may need to respond to short-term changes and/or shifts in demand. The popularity of online and digital services with younger and more educated people has implications for health equity.

PROSPERO REGISTRATION NUMBER

CRD42018093564.

Digital and online symptom checkers and assessment services for urgent care to inform a new digital platform: a systematic review

Background

Digital and online symptom checkers and assessment services are used by patients seeking guidance about health problems. NHS England is planning to introduce a digital platform (NHS111 Online) to operate alongside the NHS111 urgent-care telephone service. This review focuses on digital and online symptom checkers for urgent health problems.

Objectives

This systematic review was commissioned to provide NHS England with an independent review of previous research in this area to inform strategic decision-making and service design.

Data sources

Focused searches of seven bibliographic databases were performed and supplemented by phrase searching for names of symptom checker systems and citation searches of key included studies. The bibliographic databases searched were MEDLINE, EMBASE, The Cochrane Library, CINAHL (Cumulative Index to Nursing and Allied Health Literature), HMIC (Health Management Information Consortium), Web of Science and the Association of Computing Machinery (ACM) Digital Library, from inception up to April 2018.

Review methods

Brief inclusion criteria were (1) population – general population seeking information online or digitally to address an urgent health problem; (2) intervention – any online or digital service designed to assess symptoms, provide health advice and direct patients to appropriate services; and (3) comparator – telephone or face-to-face assessment, comparative performance in tests or simulations (studies with no comparator were included if they reported relevant outcomes). Outcomes of interest included safety, clinical effectiveness, costs or cost-effectiveness, diagnostic and triage accuracy, use of and contacts with health services, compliance with advice received, patient/carer satisfaction, and equity and inclusion. Inclusion was not restricted by study design. Screening studies for inclusion, data extraction and quality assessment were carried out by one reviewer with a sample checked for accuracy and consistency. Final decisions on study inclusion were taken by consensus of the review team. A narrative synthesis of the included studies was performed and structured around the predefined research questions and key outcomes. The overall strength of evidence for each outcome was classified as ‘stronger’, ‘weaker’, ‘conflicting’ or ‘insufficient’, based on study numbers and design.

Results

In total, 29 publications describing 27 studies were included. Studies were diverse in their design and methodology. The overall strength of the evidence was weak because it was largely based on observational studies and with a substantial component of non-peer-reviewed grey literature. There was little evidence to suggest that symptom checkers are unsafe, but studies evaluating their safety were generally short term and small scale. Diagnostic accuracy was highly variable between different systems but was generally low. Algorithm-based triage tended to be more risk averse than that of health professionals. Inconsistent evidence was found on effects on service use. There was very limited evidence on patients’ reactions to online triage advice. The studies showed that younger and more highly educated people are more likely to use these services. Study participants generally expressed high levels of satisfaction with digital and online triage services, albeit in uncontrolled studies.

Limitations

Findings from symptom checker systems for specific conditions may not be applicable to more general systems and vice versa. Studies of symptom checkers as part of electronic consultation systems in general practice were also included, which is a slightly different setting from a general ‘digital 111’ service. Most studies were screened by one reviewer.

Conclusions

Major uncertainties surround the probable impact of digital 111 services on most outcomes. It will be important to monitor and evaluate the services using all available data sources and by commissioning high-quality research.

Future work

Priorities for research include comparisons of different systems, rigorous economic evaluations and investigations of patient pathways.

Study registration

The study is registered as PROSPERO CRD42018093564.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Web Use for Symptom Appraisal of Physical Health Conditions: A Systematic Review

Background

The Web has become an important information source for appraising symptoms. We need to understand the role it currently plays in help seeking and symptom evaluation to leverage its potential to support health care delivery.

Objective

The aim was to systematically review the literature currently available on Web use for symptom appraisal.

Methods

We searched PubMed, EMBASE, PsycINFO, ACM Digital Library, SCOPUS, and Web of Science for any empirical studies that addressed the use of the Web by lay people to evaluate symptoms for physical conditions. Articles were excluded if they did not meet minimum quality criteria. Study findings were synthesized using a thematic approach.

Results

A total of 32 studies were included. Study designs included cross-sectional surveys, qualitative studies, experimental studies, and studies involving website/search engine usage data. Approximately 35% of adults engage in Web use for symptom appraisal, but this proportion varies between 23% and 75% depending on sociodemographic and disease-related factors. Most searches were symptom-based rather than condition-based. Users viewed only the top search results and interacted more with results that mentioned serious conditions. Web use for symptom appraisal appears to impact on the decision to present to health services, communication with health professionals, and anxiety.

Conclusions

Web use for symptom appraisal has the potential to influence the timing of help seeking for symptoms and the communication between patients and health care professionals during consultations. However, studies lack suitable comparison groups as well as follow-up of participants over time to determine whether Web use results in health care utilization and diagnosis. Future research should involve longitudinal follow-up so that we can weigh the benefits of Web use for symptom appraisal (eg, reductions in delays to diagnosis) against the disadvantages (eg, unnecessary anxiety and health care use) and relate these to health care costs.

Evaluation of symptom checkers for self diagnosis and triage: audit study

OBJECTIVE

To determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).

DESIGN

Audit study.

SETTING

Publicly available, free symptom checkers.

PARTICIPANTS

23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).

MAIN OUTCOME MEASURES

For symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).

RESULTS

The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.

CONCLUSIONS

Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.

Should you search the Internet for information about your acute symptom?

OBJECTIVE

To determine if symptom-related Web sites give sufficient information for users to seek urgent care when warranted.

MATERIALS AND METHODS

We reviewed 120 Web sites (15 sites for each of eight acute symptoms). Symptom-related sites were identified with Google, Yahoo!®, and Bing™ searches and focused on potentially hazardous symptoms such as chest pain, shortness of breath, abdominal pain, and syncope. We reviewed each symptom-related site for the presence of critical symptom indicators (key symptom characteristics and associated factors) that triage the user to urgent care.

RESULTS

Of the 120 sites reviewed, 41 (33%) contained no critical symptom indicators. No site contained a complete set of critical symptom indicators. Overall, out of the 1,020 total critical symptoms searched for in the sites, we only found 329 (32%). When present, critical symptom indicators were found on the top half of the first page of the site in only 34%. Specific recommendations for further care were absent in 42% of the cases where critical symptom indicators were identified.

CONCLUSIONS

Symptom-related sites ranked highly by major search engines lack much of the information needed to make a decision about whether a symptom needs urgent attention. When present, this information is usually not located where users can rapidly access it and often lacks prescriptive guidance for users to seek care. Until more sites contain at least minimal triage advice, relying on an Internet search to help determine the urgency of a symptom could be risky.