Performance, Acceptability, and Usability of Respiratory Rate Timers and Pulse Oximeters When Used by Frontline Health Workers to Detect Symptoms of Pneumonia in Sub-Saharan Africa and Southeast Asia: Protocol for a Two-Phase, Multisite, Mixed-Methods Trial

Assessing the acceptability of technological health innovations in sub-Saharan Africa: a scoping review and a best fit framework synthesis

Acceptability is a key concept used to analyze the introduction of a health innovation in a specific setting. However, there seems to be a lack of clarity in this notion, both conceptually and practically. In low and middle-income countries, programs to support the diffusion of new technological tools are multiplying. They face challenges and difficulties that need to be understood with an in-depth analysis of the acceptability of these innovations. We performed a scoping review to explore the theories, methods and conceptual frameworks that have been used to measure and understand the acceptability of technological health innovations in sub-Saharan Africa. The review confirmed the lack of common definitions, conceptualizations and practical tools addressing the acceptability of health innovations. To synthesize and combine evidence, both theoretically and empirically, we then used the "best fit framework synthesis" method. Based on five conceptual and theoretical frameworks from scientific literature and evidence from 33 empirical studies, we built a conceptual framework in order to understand the acceptability of technological health innovations. This framework comprises 6 determinants (compatibility, social influence, personal emotions, perceived disadvantages, perceived advantages and perceived complexity) and two moderating factors (intervention and context). This knowledge synthesis work has also enabled us to propose a chronology of the different stages of acceptability.

Repeatability of Pulse Oximetry Measurements in Children During Triage in 2 Ugandan Hospitals

BACKGROUND

In low- and middle-income countries, health workers use pulse oximeters for intermittent spot measurements of oxygen saturation (SpO2). However, the accuracy and reliability of pulse oximeters for spot measurements have not been determined. We evaluated the repeatability of spot measurements and the ideal observation time to guide recommendations during spot check measurements.

METHODS

Two 1-minute measurements were taken for the 3,903 subjects enrolled in the study conducted April 2020-January 2022 in Uganda, collecting 1 Hz SpO2 and signal quality index (SQI) data. The repeatability between the 2 measurements was assessed using an intraclass correlation coefficient (ICC), calculated using a median of all seconds of non-zero SpO2 values for each recording (any quality, Q1) and again with a quality filter only using seconds with SQI 90% or higher (good quality, Q2). The ICC was also recalculated for both conditions of Q1 and Q2 using the initial 5 seconds, then the initial 10 seconds, and continuing with 5-second increments up to the full 60 seconds. Lastly, the whole minute ICC was calculated with good quality (Q2), including only records where both measurements had a mean SQI of more than 70% (Q3).

RESULTS

The repeatability ICC with condition Q1 was 0.591 (95% confidence interval [CI]=0.570, 0.611). Using only the first 5 seconds of each measurement reduced the repeatability to 0.200 (95% CI=0.169, 0.230). Filtering with Q2, the whole-minute ICC was 0.855 (95% CI=0.847, 0.864). The ICC did not improve beyond the first 35 seconds. For Q3, the repeatability rose to 0.908 (95% CI=0.901, 0.914).

CONCLUSIONS

Training guidelines must emphasize the importance of signal quality and duration of measurement, targeting a minimum of 35 seconds of adequate-quality, stable data. In addition, the design of new devices should incorporate user prompts and force quality checks to encourage more accurate pulse oximetry measurements.

Usability of pulse oximeters used by community health and primary care workers as screening tools for severe illness in children under five in low resource settings: A cross-sectional study in Cambodia, Ethiopia, South Sudan, and Uganda

Timely recognition and referral of severely ill children is especially critical in low-resource health systems. Pulse-oximeters can improve health outcomes of children by detecting hypoxaemia, a severity indicator of the most common causes of death in children. Cost-effectiveness of pulse-oximeters has been proven in low-income settings. However, evidence on their usability in community health settings is scarce.This study explores the usability of pulse-oximeters for community health and primary care workers in Cambodia, Ethiopia, South Sudan, and Uganda. We collected observational data, through a nine-task checklist, and survey data, using a five-point Likert scale questionnaire, capturing three usability aspects (effectiveness, efficiency, and satisfaction) of single-probe fingertip and multi-probe handheld devices. Effectiveness was determined by checklist completion rates and task completion rates per checklist item. Efficiency was reported as proportion of successful assessments within three attempts. Standardized summated questionnaire scores (min = 0, max = 100) determined health worker's satisfaction. Influencing factors on effectiveness and satisfaction were explored through hypothesis tests between independent groups (device type, cadre of health worker, country). Checklist completion rate was 78.3% [CI 72.6-83.0]. Choosing probes according to child age showed the lowest task completion rate of 68.7% [CI 60.3%-76.0%]. In 95.6% [CI 92.7%-97.4%] of assessments a reading was obtained within three attempts. The median satisfaction score was 95.6 [IQR = 92.2-99.0]. Significantly higher checklist completion rates were observed with single-probe fingertip devices (p<0.001) and children 12-59 months (p<0.001). We found higher satisfaction scores in South Sudan (p<0.001) and satisfaction varied slightly between devices. From a usability perspective single-probe devices for all age groups should be prioritized for scaled implementation. Further research on easy to use and accurate devices for infants is much needed.

Performance and usability of a new mobile application for measuring respiratory rate in young children with acute lower respiratory infections

OBJECTIVES

Respiratory rate (RR) measurement is critical to diagnosing pneumonia in resource-constrained settings, but accurate RR measurement is challenging. The acute lower respiratory illness treatment and evaluation (ALRITE) mobile phone application (app), designed to help healthcare workers (HCWs) manage pediatric respiratory illnesses, includes a semiautomated RR counter. This study aimed to evaluate the accuracy and usability of the ALRITE RR counter and a commercially available RR counter app, RRate, with a reference standard.

METHODS

This was a cross-sectional observational study of HCWs. Participants used both apps to measure the RR of pediatric patients from standardized videos. The reference standard was determined by consensus of a manual 1-min count by two providers. We assessed agreement using Spearman's rank correlation coefficient and constructed Bland-Altman plots to determine bias and limits of agreement. Participants completed a usability survey.

RESULTS

Thirty-nine HCWs participated. The agreement between the apps and reference standard (Spearman's coefficient) was 0.83 (95% confidence interval [CI]: 0.78-0.87) for ALRITE and 0.62 (95% CI: 0.52-0.70) for RRate. ALRITE had a bias of -2 breaths/min (lower limit of agreement [LoA] -16 to +12) and RRate had a bias of -0.4 breaths/min (LoA -24 to +23) compared to the reference standard. Both apps had a poorer agreement at higher RRs. Based on usability survey responses, 95% found ALRITE easy to use.

CONCLUSIONS

The ALRITE RR counter has acceptable accuracy for counting RR in infants with respiratory distress, appears to be more accurate than a commercially available option, and was user-friendly. The ALRITE RR counter is a promising tool for meriting evaluation in real-world settings.

Stock-outs of essential medicines among community health workers (CHWs) in low- and middle-income countries (LMICs): a systematic literature review of the extent, reasons, and consequences

BACKGROUND

This paper explores the extent of community-level stock-out of essential medicines among community health workers (CHWs) in low- and middle-income countries (LMICs) and identifies the reasons for and consequences of essential medicine stock-outs.

METHODS

A systematic review was conducted and reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Five electronic databases were searched with a prespecified strategy and the grey literature examined, January 2006-March 2021. Papers containing information on (1) the percentage of CHWs stocked out or (2) reasons for stock-outs along the supply chain and consequences of stock-out were included and appraised for risk of bias. Outcomes were quantitative data on the extent of stock-out, summarized using descriptive statistics, and qualitative data regarding reasons for and consequences of stock-outs, analyzed using thematic content analysis and narrative synthesis.

RESULTS

Two reviewers screened 1083 records; 78 evaluations were included. Over the last 15 years, CHWs experienced stock-outs of essential medicines nearly one third of the time and at a significantly (p < 0.01) higher rate than the health centers to which they are affiliated (28.93% [CI 95%: 28.79-29.07] vs 9.17% [CI 95%: 8.64-9.70], respectively). A comparison of the period 2006-2015 and 2016-2021 showed a significant (p < 0.01) increase in CHW stock-out level from 26.36% [CI 95%: 26.22-26.50] to 48.65% [CI 95%: 48.02-49.28] while that of health centers increased from 7.79% [95% CI 7.16-8.42] to 14.28% [95% CI 11.22-17.34]. Distribution barriers were the most cited reasons for stock-outs. Ultimately, patients were the most affected: stock-outs resulted in out-of-pocket expenses to buy unavailable medicines, poor adherence to medicine regimes, dissatisfaction, and low service utilization.

CONCLUSIONS

Community-level stock-out of essential medicines constitutes a serious threat to achieving universal health coverage and equitable improvement of health outcomes. This paper suggests stock-outs are getting worse, and that there are particular barriers at the last mile. There is an urgent need to address the health and non-health system constraints that prevent the essential medicines procured for LMICs by international and national stakeholders from reaching the people who need them the most.

Prevalence of hypoxaemia in children with pneumonia in low-income and middle-income countries: a systematic review and meta-analysis

Background

Pneumonia accounts for around 15% of all deaths of children younger than 5 years globally. Most happen in resource-constrained settings and are potentially preventable. Hypoxaemia is one of the strongest predictors of these deaths. We present an updated estimate of hypoxaemia prevalence among children with pneumonia in low-income and middle-income countries.

Methods

We conducted a systematic review using the following key concepts “children under five years of age” AND “pneumonia” AND “hypoxaemia” AND “low- and middle-income countries” by searching in 11 bibliographic databases and citation indices. We included all articles published between Nov 1, 2008, and Oct 8, 2021, based on observational studies and control arms of randomised and non-randomised controlled trials. We excluded protocol papers, articles reporting hypoxaemia prevalence based on less than 100 pneumonia cases, and articles published before 2008 from the review. Quality appraisal was done with the Joanna Briggs Institute tools. We reported pooled prevalence of hypoxaemia (SpO₂ <90%) by classification of clinical severity and by clinical settings by use of the random-effects meta-analysis models. We combined our estimate of the pooled prevalence of pneumonia with a previously published estimate of the number of children admitted to hospital due to pneumonia annually to calculate the total annual number of children admitted to hospital with hypoxaemic pneumonia.

Findings

We identified 2825 unique records from the databases, of which 57 studies met the eligibility criteria: 26 from Africa, 23 from Asia, five from South America, and four from multiple continents. The prevalence of hypoxaemia was 31% (95% CI 26–36; 101 775 children) among all children with WHO-classified pneumonia, 41% (33–49; 30 483 children) among those with very severe or severe pneumonia, and 8% (3–16; 2395 children) among those with non-severe pneumonia. The prevalence was much higher in studies conducted in emergency and inpatient settings than in studies conducted in outpatient settings. In 2019, we estimated that over 7 million children (95% CI 5–8 million) were admitted to hospital with hypoxaemic pneumonia. The studies included in this systematic review had high τ² (ie, 0·17), indicating a high level of heterogeneity between studies, and a high I² value (ie, 99·6%), indicating that the heterogeneity was not due to chance. This study is registered with PROSPERO, CRD42019126207.

Interpretation

The high prevalence of hypoxaemia among children with severe pneumonia, particularly among children who have been admitted to hospital, emphasises the importance of overall oxygen security within the health systems of low-income and middle-income countries, particularly in the context of the COVID-19 pandemic. Even among children with non-severe pneumonia that is managed in outpatient and community settings, the high prevalence emphasises the importance of rapid identification of hypoxaemia at the first point of contact and referral for appropriate oxygen therapy.

Funding

UK National Institute for Health Research (Global Health Research Unit on Respiratory Health [RESPIRE]; 16/136/109).

Performance of five pulse oximeters to detect hypoxaemia as an indicator of severe illness in children under five by frontline health workers in low resource settings - A prospective, multicentre, single-blinded, trial in Cambodia, Ethiopia, South Sudan, and Uganda

BACKGROUND

Low blood oxygen saturation (SpO₂), or hypoxaemia, is an indicator of severe illness in children. Pulse oximetry is a globally accepted, non-invasive method to identify hypoxaemia, but rarely available outside higher-level facilities in resource-constrained countries. This study aims to evaluate the performance of different types of pulse oximeters amongst frontline health workers in Cambodia, Ethiopia, South Sudan, and Uganda.

METHODS

Five pulse oximeters (POx) which passed laboratory testing, out of an initial 32 potential pulse oximeters, were evaluated by frontline health workers for performance, defined as agreement between the SpO₂ measurements of the test device and the reference standard. The study protocol is registered with the Australia New Zealand Clinical Trials Registry (Ref: ACTRrn12615000348550).

FINDINGS

Two finger-tip pulse oximeters (Contec and Devon), two handheld pulse oximeters (Lifebox and Utech), and one phone pulse oximeter (Masimo) passed the laboratory testing. They were evaluated for performance on 1,313 children under five years old by 207 frontline health workers between February and May 2015. Phone and handheld pulse oximeters had greater overall agreement with the reference standard (56%; 95% CI 0.52 - 0.60 to 68%; 95% CI 0.65 - 0.71) than the finger-tip POx (31%; 95% CI 0.26 to 0.36 and 47%; 95% CI 0.42 to 0.52). Fingertip POx performance was substantially lower in the 0-2 month olds; having just 17% and 25% agreement. The finger-tip devices more often underreported SpO₂ readings (mean difference -7.9%; 95%CI -8.6,-7.2 and -3.9%; 95%CI -4.4,-3.4), and therefore over diagnosed hypoxaemia in the children assessed.

INTERPRETATION

While the Masimo phone pulse oximeter performed best, all handheld POx with age-specific probes performed well in the hands of frontline health workers, further highlighting their suitability as a screening tool of severe illness. The poor performance of the fingertip POx suggests they should not be used in children under five by frontline health workers. It is essential that POx are performance tested on children in routine settings (in vivo), not only in laboratories or controlled settings (in vitro), before being introduced at scale.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].

Performance of Four Respiratory Rate Counters to Support Community Health Workers to Detect the Symptoms of Pneumonia in Children in Low Resource Settings: A Prospective, Multicentre, Hospital-Based, Single-Blinded, Comparative Trial

BACKGROUND

Pneumonia is one of the leading causes of death in children under-five globally. The current diagnostic criteria for pneumonia are based on increased respiratory rate (RR) or chest in-drawing in children with cough and/or difficulty breathing. Accurately counting RR is difficult for community health workers (CHWs). Current RR counting devices are frequently inadequate or unavailable. This study analysed the performance of improved RR timers for detection of pneumonia symptoms in low-resource settings.

METHODS

Four RR timers were evaluated on 454 children, aged from 0 to 59 months with cough and/or difficulty breathing, over three months, by CHWs in hospital settings in Cambodia, Ethiopia, South Sudan and Uganda. The devices were the Mark Two ARI timer (MK2 ARI), counting beads with ARI timer, Rrate Android phone and the Respirometer feature phone applications. Performance was evaluated for agreement with an automated RR reference standard (Masimo Root patient monitoring and connectivity platform with ISA CO2 capnography). This study is registered with ANZCTR [ACTRN12615000348550].

FINDINGS

While most CHWs managed to achieve a RR count with the four devices, the agreement was low for all; the mean difference of RR measurements from the reference standard for the four devices ranged from 0.5 (95% C.I. - 2.2 to 1.2) for the respirometer to 5.5 (95% C.I. 3.2 to 7.8) for Rrate. Performance was consistently lower for young infants (0 to < 2 months) than for older children (2 to ≤ 59 months). Agreement of RR classification into fast and normal breathing was moderate across all four devices, with Cohen's Kappa statistics ranging from 0.41 (SE 0.04) to 0.49 (SE 0.05).

INTERPRETATION

None of the four devices evaluated performed well based on agreement with the reference standard. The ARI timer currently recommended for use by CHWs should only be replaced by more expensive, equally performing, automated RR devices when aspects such as usability and duration of the device significantly improve the patient-provider experience.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].