Performance of five pulse oximeters to detect hypoxaemia as an indicator of severe illness in children under five by frontline health workers in low resource settings - A prospective, multicentre, single-blinded, trial in Cambodia, Ethiopia, South Sudan, and Uganda

BACKGROUND

Low blood oxygen saturation (SpO₂), or hypoxaemia, is an indicator of severe illness in children. Pulse oximetry is a globally accepted, non-invasive method to identify hypoxaemia, but rarely available outside higher-level facilities in resource-constrained countries. This study aims to evaluate the performance of different types of pulse oximeters amongst frontline health workers in Cambodia, Ethiopia, South Sudan, and Uganda.

METHODS

Five pulse oximeters (POx) which passed laboratory testing, out of an initial 32 potential pulse oximeters, were evaluated by frontline health workers for performance, defined as agreement between the SpO₂ measurements of the test device and the reference standard. The study protocol is registered with the Australia New Zealand Clinical Trials Registry (Ref: ACTRrn12615000348550).

FINDINGS

Two finger-tip pulse oximeters (Contec and Devon), two handheld pulse oximeters (Lifebox and Utech), and one phone pulse oximeter (Masimo) passed the laboratory testing. They were evaluated for performance on 1,313 children under five years old by 207 frontline health workers between February and May 2015. Phone and handheld pulse oximeters had greater overall agreement with the reference standard (56%; 95% CI 0.52 - 0.60 to 68%; 95% CI 0.65 - 0.71) than the finger-tip POx (31%; 95% CI 0.26 to 0.36 and 47%; 95% CI 0.42 to 0.52). Fingertip POx performance was substantially lower in the 0-2 month olds; having just 17% and 25% agreement. The finger-tip devices more often underreported SpO₂ readings (mean difference -7.9%; 95%CI -8.6,-7.2 and -3.9%; 95%CI -4.4,-3.4), and therefore over diagnosed hypoxaemia in the children assessed.

INTERPRETATION

While the Masimo phone pulse oximeter performed best, all handheld POx with age-specific probes performed well in the hands of frontline health workers, further highlighting their suitability as a screening tool of severe illness. The poor performance of the fingertip POx suggests they should not be used in children under five by frontline health workers. It is essential that POx are performance tested on children in routine settings (in vivo), not only in laboratories or controlled settings (in vitro), before being introduced at scale.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].

Performance of Four Respiratory Rate Counters to Support Community Health Workers to Detect the Symptoms of Pneumonia in Children in Low Resource Settings: A Prospective, Multicentre, Hospital-Based, Single-Blinded, Comparative Trial

BACKGROUND

Pneumonia is one of the leading causes of death in children under-five globally. The current diagnostic criteria for pneumonia are based on increased respiratory rate (RR) or chest in-drawing in children with cough and/or difficulty breathing. Accurately counting RR is difficult for community health workers (CHWs). Current RR counting devices are frequently inadequate or unavailable. This study analysed the performance of improved RR timers for detection of pneumonia symptoms in low-resource settings.

METHODS

Four RR timers were evaluated on 454 children, aged from 0 to 59 months with cough and/or difficulty breathing, over three months, by CHWs in hospital settings in Cambodia, Ethiopia, South Sudan and Uganda. The devices were the Mark Two ARI timer (MK2 ARI), counting beads with ARI timer, Rrate Android phone and the Respirometer feature phone applications. Performance was evaluated for agreement with an automated RR reference standard (Masimo Root patient monitoring and connectivity platform with ISA CO₂ capnography). This study is registered with ANZCTR [ACTRN12615000348550].

FINDINGS

While most CHWs managed to achieve a RR count with the four devices, the agreement was low for all; the mean difference of RR measurements from the reference standard for the four devices ranged from 0.5 (95% C.I. - 2.2 to 1.2) for the respirometer to 5.5 (95% C.I. 3.2 to 7.8) for Rrate. Performance was consistently lower for young infants (0 to < 2 months) than for older children (2 to ≤ 59 months). Agreement of RR classification into fast and normal breathing was moderate across all four devices, with Cohen's Kappa statistics ranging from 0.41 (SE 0.04) to 0.49 (SE 0.05).

INTERPRETATION

None of the four devices evaluated performed well based on agreement with the reference standard. The ARI timer currently recommended for use by CHWs should only be replaced by more expensive, equally performing, automated RR devices when aspects such as usability and duration of the device significantly improve the patient-provider experience.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].

Performance, Acceptability, and Usability of Respiratory Rate Timers and Pulse Oximeters When Used by Frontline Health Workers to Detect Symptoms of Pneumonia in Sub-Saharan Africa and Southeast Asia: Protocol for a Two-Phase, Multisite, Mixed-Methods Trial

Background

Pneumonia is one of the leading causes of death in children aged under 5 years in both sub-Saharan Africa and Southeast Asia. The current diagnostic criterion for pneumonia is based on the increased respiratory rate (RR) in children with cough or difficulty breathing. Low oxygen saturation, measured using pulse oximeters, is indicative of severe pneumonia. Health workers often find it difficult to accurately count the number of breaths, and the current RR counting devices are often difficult to use or unavailable. Nonetheless, improved counting devices and low-cost pulse oximeters are now available on the market.

Objective

The objective of our study was to identify the most accurate, usable, and acceptable devices for the diagnosis of pneumonia symptoms by community health workers and first-level health facility workers or frontline health workers in resource-poor settings.

Methods

This was a multicenter, prospective, two-stage, observational study to assess the performance and usability or acceptability of 9 potential diagnostic devices when used to detect symptoms of pneumonia in the hands of frontline health workers. Notably, 188 possible devices were ranked and scored, tested for suitability in a laboratory, and 5 pulse oximeters and 4 RR timers were evaluated for usability and performance by frontline health workers in hospital, health facility, and community settings. The performance was evaluated against 2 references over 3 months in Cambodia, Ethiopia, South Sudan, and Uganda. Furthermore, acceptability and usability was subsequently evaluated using both qualitative and quantitative methodologies in routine practice, over 3 months, in the 4 countries.

Results

This project was funded in 2014, and data collection has been completed. Data analysis is currently under way, and the first results are expected to be submitted for publication in 2018.

Conclusions

This is the first large-scale evaluation of tools to detect symptoms of pneumonia at the community level. In addition, selecting an appropriate reference standard against which the devices were measured was challenging given the lack of existing standards and differences of opinions among experts. The findings from this study will help create a standardized and validated protocol for future studies and support further comparative testing of diagnostic devices in these settings.

Trial Registration

Australian New Zealand Clinical Trials Registry ACTRN12615000348550; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=367306&isReview=true (Archived by Website at http://www.webcitation.org/72OcvgBcf)

International Registered Report Identifier (IRRID)

RR1-10.2196/10191

Childhood pneumonia diagnostics: community health workers' and national stakeholders' differing perspectives of new and existing aids

Background: Pneumonia heavily contributes to global under-five mortality. Many countries use community case management to detect and treat childhood pneumonia. Community health workers (CHWs) have limited tools to help them assess signs of pneumonia. New respiratory rate (RR) counting devices and pulse oximeters are being considered for this purpose.

Objective: To explore perspectives of CHWs and national stakeholders regarding the potential usability and scalability of seven devices to aid community assessment of pneumonia signs.

Design: Pile sorting was conducted to rate the usability and scalability of 7 different RR counting aids and pulse oximeters amongst 16 groups of participants. Following each pile-sorting session, a focus group discussion (FGD) explored participants’ sorting rationale. Purposive sampling was used to select CHWs and national stakeholders with experience in childhood pneumonia and integrated community case management (iCCM) in Cambodia, Ethiopia, Uganda and South Sudan. Pile-sorting data were aggregated for countries and participant groups. FGDs were audio recorded and transcribed verbatim. Translated FGDs transcripts were coded in NVivo 10 and analysed using thematic content analysis. Comparative analysis was performed between countries and groups to identify thematic patterns.

Results: CHWs and national stakeholders across the four countries perceived the acute respiratory infection (ARI) timer and fingertip pulse oximeter as highly scalable and easy for CHWs to use. National stakeholders were less receptive to new technologies. CHWs placed greater priority on device acceptability to caregivers and children. Both groups felt that heavy reliance on electricity reduced potential scalability and usability in rural areas. Device simplicity, affordability and sustainability were universally valued.

Conclusions: CHWs and national stakeholders prioritise different device characteristics according to their specific focus of work. The views of all relevant stakeholders, including health workers, policy makers, children and parents, should be considered in future policy decisions, research and development regarding suitable pneumonia diagnostic aids for community use.