Is the NICE traffic light system fit-for-purpose for children presenting with undifferentiated acute illness in primary care?

Supporting parents with acutely ill children: Environment scan and user evaluation of mobile applications (the SuPa kids project)

BACKGROUND

Children are one of the biggest users of emergency departments in the UK, sometimes utilising services when experiencing acute illnesses that can be managed at home. mHealth can be an efficacious way for parents/guardians to manage acute illnesses. It has also become a ubiquitous resource to promote public health interests within a resource constrained health system. However, to be effective, co-design is necessary with concepts such as usefulness, ease of use, trustworthiness and security underpinning the utility of mHealth interventions for acute childhood illness.

OBJECTIVE

The objective of this study was to conduct an environmental scan of mHealth apps for parental assessment and management of acute childhood illnesses in 0- to 5-year-olds, review the content and functionality of each identified app against the list of desired attributes identified in previous research and synthesise publicly available reviews from end users to demonstrate the quality of current apps in the marketplace and identify areas for improvement.

METHOD

App characteristics were mapped to an a priori defined list of desirable attributes. End user reviews were mapped to a modified Technology Acceptance Model framework that included overarching themes of: Perceived Usefulness (PU), Perceived Ease of Use (PEU), Perceived Trustworthiness (PT) and Perceived Security (PS) using reflexive thematic analysis.

RESULTS

Overall, 15 apps were included in the analysis (Table 2). Fourteen of the 15 apps were available on both the iOS and Android platforms. 'Baby Illness, Prevention, Cure' was only available on the Android store. All apps were free to download and did not restrict access to information or features via in-app purchases. Baby and child first aid (British Red Cross) was the earliest app to be released (2014). This app was updated annually, with the most recent update being 2022. The most recent apps to be released (2018) were 'AskFirst' (formerly Ask NHS), 'Essex Child Health' and 'The Lullaby Trust Baby Check', updated in 2022, 2020 and 2019, respectively. No individual app met all the desirable attributes highlighted by parents in previous research. Both 'Healthier Together', and 'The Lullaby Trust Baby Check' included all but one desirable attributes (video), as did the ''CATCH (Common Approach to Children's Health), which did not utilise a traffic light system. Apps that were locality specific were not rated by users (Berkshire Child Health, Cheshire Child Health, Child Health Guide Newham, Child Health HMR, Oldham Child Illness and Walsall Healthy Child). All other apps were rated from 2/5 to 5/5 stars. When considering localities, the 'CATCH' app was only supported in Halton, Cheshire, Knowsley, Liverpool, St Helens and Vale Royal. Further investigation evidenced no correlation between inclusion of the desirable attributes and app rating.

CONCLUSION

Overall, this research has highlighted several aspects of best practise when developing mHealth apps for the management of acute childhood illnesses, for increasing PU, PEU, PT and PS; and also, places importance on co-design.

Accuracy of the NICE Traffic Light system for detecting serious illness in acutely unwell children presenting to general practice: a retrospective cohort study

Background

The National Institute for Health and Care Excellence (NICE) traffic light system was created to facilitate the assessment of unwell children in primary care. To the authors’ knowledge, no studies have validated this tool in UK general practice.

Aim

To evaluate the accuracy of this system for detecting serious illness in children presenting to general practice.

Design and setting

A retrospective diagnostic accuracy study was undertaken, using a cohort of acutely unwell children aged <5 years presenting to general practice in England and Wales.

Method

The traffic light categories of 6703 children were linked with hospital data to identify admissions and diagnoses. The sensitivity and specificity of these categories were calculated against the reference standard: a hospital-diagnosed serious illness within 7 days of GP consultation, measured using International Classification of Diseases, 10th Revision codes.

Results

In total, 2116 (31.6%) children were categorised as ‘red’; 4204 (62.7%) as ‘amber’; and 383 (5.7%) as ‘green’. There were 139 (2.1%) children who were admitted to hospital within 7 days of consultation, of whom 17 (12.2%; 0.3% overall) had a serious illness. The sensitivity of the red category (versus amber and green) was 58.8% (95% confidence interval [CI] = 32.9 to 81.6) and the specificity 68.5% (95% CI = 67.4 to 69.6). The sensitivity and specificity of red and amber combined (versus green) was 100% (95% CI = 80.5 to 100) and 5.7% (95% CI = 5.2 to 6.3), respectively.

Conclusion

The NICE traffic light system did not accurately detect children admitted with a serious illness, nor those not seriously ill who could have been managed at home. This system is not suitable for use as a clinical tool in general practice. Further research is required to update or replace the system.