Variability of respiratory rate measurements in children suspected with non-severe pneumonia in north-east Tanzania

From biorepositories to data repositories: Open-access resources accelerate early R&D and validation of equitable diagnostic tools

Diagnostics are critical tools that guide clinical decision-making for patient care and support disease surveillance. Despite its importance, developers and manufacturers often note that access to specimen panels and essential reagents is one of the key challenges in developing quality diagnostics, particularly in low-resource settings. A recent example, as the COVID-19 pandemic unfolded there was a need for clinical samples across the globe to support the rapid development of diagnostics. To address these challenges and gaps, PATH, a global nonprofit, along with its partners collaborated to create a COVID-19 biorepository to improve access to biological samples. Since then, the need for data resources to advance universal rapid diagnostic test (RDT) readers and noninvasive clinical measurement tools for screening children have also been identified and initiated. From biospecimens to data files, there are more similarities than differences in creating open-access repositories. And to ensure equitable technologies are developed, diverse sample panels and datasets are critical in the development process. Here we share one experience in creating open-access repositories as a case study to describe the steps taken, the key factors required to establish a biorepository, the ethical and legal frameworks that guided the initiative and the lessons learned. As diagnostic tools are evolving, more forms of data are critical to de-risk and accelerate early research and development (R&D) for products serving low resource settings. Creating physical and virtual repositories of freely available, well characterized, and high quality clinical and electronic data resources defray development costs to improve equitable access and test affordability.

Accuracy of shorter respiratory rate measurement times in the pediatric population

BACKGROUND

As one of the vital signs, the respiratory rate is an important index of general health in an initial examination. The duration of respiratory rate measurements is known to influence the results in adults. We examined the difference in respiratory rate measurements between two measurement durations in a pediatric population.

METHODS

The prospective, cross-sectional study was conducted from November 2017 through March 2018 at Tokyo Metropolitan Children's Medical Center and analyzed differences in the respiratory rate in a 1 min group and 30 s group using the Bland-Altman plot.

RESULTS

Ninety-five patients were enrolled. The median age was 0.99 years, and 50 patients were male. The correlation between the results of the 30 s and 1 min measurement durations was good (r²  = 0.970, P < 0.001; Spearman's rank correlation). The mean difference between the 30 s and 1 min measurement results was 0.86 (95% confidence interval: 0.27-1.45).

CONCLUSIONS

The shorter measurement duration may result in overestimation of the respiratory rate, especially in young children or children with tachypnea. Nonetheless, the shorter measurement times correlated well with the 1 min measurement, and the difference was less than one per minute compared with the 1 min measurement. Thus, this method may be able to detect life-threatening conditions earlier than longer measurement times.

Evaluating the Interrater Agreement and Acceptability of a New Reference Tool for Assessing Respiratory Rate in Children under Five with Cough and/or Difficulty Breathing

BACKGROUND

Manual assessment of respiratory rate (RR) in children is unreliable, but remains the main method to diagnose pneumonia in low-resource settings. While automated RR counters offer a potential solution, there is currently no gold standard to validate these diagnostic aids. A video-based reference tool is proposed that allows users to annotate breaths and distortions including movement periods, allowing the exclusion of distortions from the computation of RR measures similar to how new diagnostic aids account for distortions automatically. This study evaluated the interrater agreement and acceptability of the new reference tool.

METHODS

Annotations were based on previously recorded reference videos of children under five years old with cough and/or difficulty breathing (n = 50). Five randomly selected medical experts from a panel of ten annotated each video. RR measures (breaths per minute, bpm) were computed as the number of annotated certain breaths divided by the length of calm periods after removing annotated distorted periods.

RESULTS

Reviewers showed good interrater agreement on continuous RR {standard error of measurement (SEM) [4.8 (95%CI 4.4-5.3)]} and substantial agreement on classification of fast breathing (Fleiss kappa, κ 0.71). Agreement was lowest in the youngest age group [< 2 months: SEM 6.2 (5.4-7.4) bpm, κ 0.48; 2-11 months: 4.7 (4.0-5.8) bpm, κ 0.84; 12-59 months: 2.6 (2.2-3.1) bpm, κ 0.8]. Reviewers found the functionalities of the tool helpful in annotating breaths, but remained uncertain about the validity of their annotations.

CONCLUSIONS

Before the new tool can be considered a reference standard for RR assessments, interrater agreement in children younger than 2 months must be improved.

Usability and acceptability of a multimodal respiratory rate and pulse oximeter device in case management of children with symptoms of pneumonia: A cross-sectional study in Ethiopia

Aim

Pneumonia is the leading infectious cause of death among children under five globally. Many pneumonia deaths result from inappropriate treatment due to misdiagnosis of signs and symptoms. This study aims to identify whether health extension workers (HEWs) in Ethiopia, using an automated multimodal device (Masimo Rad‐G), adhere to required guidelines while assessing and classifying under five children with cough or difficulty breathing and to understand device acceptability.

Methods

A cross‐sectional study was conducted in three districts of Southern Nations, Nationalities, and Peoples' Region, Ethiopia. Between September and December 2018, 133 HEWs were directly observed using Rad‐G while conducting 599 sick child consultations. Usability was measured as adherence to the World Health Organization requirements to assess fast breathing and device manufacturer instructions for use. Acceptability was assessed using semi‐structured interviews with HEWs, first‐level health facility workers and caregivers.

Results

Adherence using the Rad‐G routinely for 2 months was 85.3% (95% CI 80.2, 89.3). Health workers and caregivers stated a preference for Rad‐G. Users highlighted a number of device design issues.

Conclusion

While demonstrating high levels of acceptability and usability, the device modifications to consider include better probe fit, improved user interface with exclusive age categories and simplified classification outcomes.

The immunomodulatory effects of vitamin D drops in children with recurrent respiratory tract infections

OBJECTIVE

To investigate the effects of vitamin D drops on immune function in children with recurrent respiratory tract infections (RRTI).

METHODS

The clinical data of 119 children with RRTI in our hospital were retrospectively retrieved, and they were divided into group A (n=59, receiving routine treatment) and group B (n=60, receiving vitamin D drops) based on their treatment modality. The clinical efficacy, symptom disappearance time, immune function index, insulin-like growth factor (IGF-1), 25-hydroxyvitamin D₃ [25-(OH)D₃], serum y-interferon (INF-y), and the number of episodes of respiratory tract infections were compared between the two groups.

RESULTS

The total effective rate of treatment in group B was 96.67%, which was significantly higher than 71.19% in group A (P<0.05). Children in group B had shorter time to disappearance of lung rales, cough, and fever than group A (P<0.05). Group B had higher IgA, IgG, and IgM levels, higher CD4⁺, CD3⁺ levels and lower CD8⁺ levels as well as higher IGF-1, 25-(OH)D₃, INF-y levels, and fewer respiratory infections after treatment than group A (P<0.05).

CONCLUSION

Vitamin D drops are effective in the treatment of children with RRTI, which is beneficial to the improvement of clinical symptoms and immune function.

Social determinants of deaths from pneumonia and tuberculosis in children in Brazil: an ecological study

OBJECTIVE

To identify the risk areas of deaths due to unspecified pneumonia and tuberculosis (TB) in children, and to identify if there is a relationship between these events with higher TB incidence and social determinants.

METHODS

Ecological study carried out in Brazil. All cases of TB or unspecified pneumonia deaths in children under 5 years of age reported between 2006 and 2016 were included and collected through Department of Informatics of the Unified Health System (Brazil's electronic database). The Spatial Scan Statistics was used to identify areas at higher risk of dying from this event. The spatial association was verified through the Getis-Ord techniques. The Bivariate Moran Global Index was used to verify the spatial autocorrelation between the two events. To identify the association of TB and pneumonia deaths with endemic areas of pulmonary TB and social determinants, four explanatory statistical models were identified.

RESULTS

A total of 21 391 cases of pneumonia and 238 cases of TB were identified. Spatial scanning analysis enabled the detection of four clusters of risk for TB (relative risk, RR, between 3.30 and 18.18) and 22 clusters for pneumonia (RR between 1.38 and 5.24). The spatial association of the events was confirmed (z-score 3.74 and 64.34) and spatial autocorrelation between events (Moran Index:0.031 (p=0.001)). The zero-inflated negative binomial distribution was chosen, and an association for both events was identified with the TB incidence rate (OR 5.3, 95% CI 2.85 to 9.84; OR 6.63, 95% CI 5.62 to 7.81), with the Gini Index (OR 1.78, 95% CI 1.12 to 2.82; OR 4.22, 95% CI 3.63 to4.92). Primary care coverage showed an inverse association for both events (OR 0.10, 95% CI 0.67 to 0.17; OR 0.18, 95% CI 0.15 to 0.21) for pneumonia). Finally, a family that benefited from the Bolsa Família Programme had an inverse association for deaths from pneumonia (OR 0.81, 95% CI 0.52 to 1.25).

CONCLUSIONS

The results do not just contribute to reduce mortality in children, but mainly contribute to prevent premature deaths through identification of critical areas in Brazil, which is crucial to qualify health surveillance services.

Determining the Agreement Between an Automated Respiratory Rate Counter and a Reference Standard for Detecting Symptoms of Pneumonia in Children: Protocol for a Cross-Sectional Study in Ethiopia

Background

Acute respiratory infections (ARIs), primarily pneumonia, are the leading infectious cause of under-5 mortality worldwide. Manually counting respiratory rate (RR) for 60 seconds using an ARI timer is commonly practiced by community health workers to detect fast breathing, an important sign of pneumonia. However, correctly counting breaths manually and classifying the RR is challenging, often leading to inappropriate treatment. A potential solution is to introduce RR counters, which count and classify RR automatically.

Objective

This study aims to determine how the RR count of an Automated Respiratory Infection Diagnostic Aid (ARIDA) agrees with the count of an expert panel of pediatricians counting RR by reviewing a video of the child’s chest for 60 seconds (reference standard), for children aged younger than 5 years with cough and/or difficult breathing.

Methods

A cross-sectional study aiming to enroll 290 children aged 0 to 59 months presenting to pediatric in- and outpatient departments at a teaching hospital in Addis Ababa, Ethiopia, was conducted. Enrollment occurred between April and May 2017. Once enrolled, children participated in at least one of three types of RR evaluations: (1) agreement—measure the RR count of an ARIDA in comparison with the reference standard, (2) consistency—measure the agreement between two ARIDA devices strapped to one child, and (3) RR fluctuation—measure RR count variability over time after ARIDA attachment as measured by a manual count. The agreement and consistency of expert clinicians (ECs) counting RR for the same child with the Mark 2 ARI timer for 60 seconds was also measured in comparison with the reference standard.

Results

Primary outcomes were (1) mean difference between the ARIDA and reference standard RR count (agreement) and (2) mean difference between RR counts obtained by two ARIDA devices started simultaneously (consistency).

Conclusions

Study strengths included the design allowing for comparison between both ARIDA and the EC with the reference standard RR count. A limitation is that exactly the same set of breaths were not compared between ARIDA and the reference standard since ARIDA can take longer than 60 seconds to count RR. Also, manual RR counting, even when aided by a video of the child’s chest movements, is subject to human error and can result in low interrater reliability. Further work is needed to reach global consensus on the most appropriate reference standard and an acceptable level of agreement to provide ministries of health with evidence to make an informed decision on whether to scale up new automated RR counters.

Trial Registration

ClinicalTrials.gov NCT03067558; https://clinicaltrials.gov/ct2/show/NCT03067558

International Registered Report Identifier (IRRID)

RR1-10.2196/16531

Automated Respiratory Rate Counter to Assess Children for Symptoms of Pneumonia: Protocol for Cross-Sectional Usability and Acceptability Studies in Ethiopia and Nepal

Background

Manually counting a child’s respiratory rate (RR) for 60 seconds using an acute respiratory infection timer is the World Health Organization (WHO) recommended method for detecting fast breathing as a sign of pneumonia. However, counting the RR is challenging and misclassification of an observed rate is common, often leading to inappropriate treatment. To address this gap, the acute respiratory infection diagnostic aid (ARIDA) project was initiated in response to a call for better pneumonia diagnostic aids and aimed to identify and assess automated RR counters for classifying fast breathing pneumonia when used by front-line health workers in resource-limited community settings and health facilities. The Children’s Automated Respiration Monitor (ChARM), an automated RR diagnostic aid using accelerometer technology developed by Koninklijke Philips NV, and the Rad-G, a multimodal RR diagnostic and pulse oximeter developed by Masimo, were the two devices tested in these studies conducted in the Southern Nations, Nationalities, and Peoples’ Region in Ethiopia and in the Karnali region in Nepal.

Objective

In these studies, we aimed to understand the usability of two new automated RR diagnostic aids for community health workers (CHWs; health extension workers [Ethiopia] and female community health volunteers [Nepal]) and their acceptability to CHWs in Ethiopia and Nepal, first-level health facility workers (FLHFWs) in Ethiopia only, and caregivers in both Ethiopia and Nepal.

Methods

This was a prospective, cross-sectional study with a mixed methods design. CHWs and FLHFWs were trained to use both devices and provided with refresher training on all WHO requirements to assess fast breathing. Immediately after training, CHWs were observed using ARIDA on two children. Routine pneumonia case management consultations for children aged 5 years and younger and the device used for these consultations between the first and second consultations were recorded by CHWs in their patient log books. CHWs were observed a second time after 2 months. Semistructured interviews were also conducted with CHWs, FLHFWs, and caregivers. The proportion of consultations with children aged 5 years and younger where CHWs using an ARIDA and adhered to all WHO requirements to assess fast breathing and device manufacturer instructions for use after 2 months will be calculated. Qualitative data from semistructured interviews will be analyzed using a thematic framework approach.

Results

The ARIDA project was funded in November 2015, and data collection was conducted between April and December 2018. Data analysis is currently under way and the first results are expected to be submitted for publication in 2020.

Conclusions

This is the first time the usability and acceptability of automated RR counters in low-resource settings have been evaluated. Outcomes will be relevant for policy makers and are important for future research of this new class of diagnostic aids for the management of children with suspected pneumonia.

International Registered Report Identifier (IRRID)

DERR1-10.2196/14405

Does Chest Attachment of an Automated Respiratory Rate Monitor Influence the Actual Respiratory Rate in Children Under Five?

Pneumonia is one of the leading causes of death in children under 5 years worldwide. In resource-limited settings, WHO recommendations state that pneumonia can be presumptively diagnosed through the presence of cough and/or difficult breathing and a respiratory rate (RR) that is higher than age-specific cutoffs. As a new diagnostic aid the children's automated respiration monitor (ChARM) can automatically measure and classify RR in children under 5 years, but the effect of its chest attachment on the RR has not been studied. The aim of this study was to understand if misclassification of the true RR occurred by ChARM attachment. Two hundred eighty-seven children at a health center in South Ethiopia were screened for eligibility, with 188 children aged 2-59 months enrolled in the study. The RR was measured manually before and 1, 3, and 5 minutes after ChARM attachment. The proportion of children with fast or normal RR classification at baseline and the change between RR classifications over time were analyzed. Eight (4.9%; 95% CI 2.1, 9.4) of 163 children changed RR classification from normal to fast between the baseline RR count and the 1 minute RR count. Results from this study suggest that ChARM has a minor influence on the RR of children immediately after attachment, in most cases without clinical importance.

A Systematic Review of Tools to Measure Respiratory Rate in Order to Identify Childhood Pneumonia

Pneumonia is the leading infectious cause of death in children worldwide, with most deaths occurring in developing countries. Measuring respiratory rate is critical to the World Health Organization's guidelines for diagnosing childhood pneumonia in low-resource settings, yet it is difficult to accurately measure. We conducted a systematic review to landscape existing respiratory rate measurement technologies. We searched PubMed, Embase, and Compendex for studies published through September 2017 assessing the accuracy of respiratory rate measurement technologies in children. We identified 16 studies: 2 describing manual devices and 14 describing automated devices. Although both studies describing manual devices took place in low-resource settings, all studies describing automated devices were conducted in well-resourced settings. Direct comparison between studies was complicated by small sample size, absence of a consistent reference standard, and variations in comparison methodology. There is an urgent need for affordable and appropriate innovations that can reliably measure a child's respiratory rate in low-resource settings. Accelerating development or scale-up of these technologies could have the potential to advance childhood pneumonia diagnosis worldwide.

Performance of Four Respiratory Rate Counters to Support Community Health Workers to Detect the Symptoms of Pneumonia in Children in Low Resource Settings: A Prospective, Multicentre, Hospital-Based, Single-Blinded, Comparative Trial

BACKGROUND

Pneumonia is one of the leading causes of death in children under-five globally. The current diagnostic criteria for pneumonia are based on increased respiratory rate (RR) or chest in-drawing in children with cough and/or difficulty breathing. Accurately counting RR is difficult for community health workers (CHWs). Current RR counting devices are frequently inadequate or unavailable. This study analysed the performance of improved RR timers for detection of pneumonia symptoms in low-resource settings.

METHODS

Four RR timers were evaluated on 454 children, aged from 0 to 59 months with cough and/or difficulty breathing, over three months, by CHWs in hospital settings in Cambodia, Ethiopia, South Sudan and Uganda. The devices were the Mark Two ARI timer (MK2 ARI), counting beads with ARI timer, Rrate Android phone and the Respirometer feature phone applications. Performance was evaluated for agreement with an automated RR reference standard (Masimo Root patient monitoring and connectivity platform with ISA CO2 capnography). This study is registered with ANZCTR [ACTRN12615000348550].

FINDINGS

While most CHWs managed to achieve a RR count with the four devices, the agreement was low for all; the mean difference of RR measurements from the reference standard for the four devices ranged from 0.5 (95% C.I. - 2.2 to 1.2) for the respirometer to 5.5 (95% C.I. 3.2 to 7.8) for Rrate. Performance was consistently lower for young infants (0 to < 2 months) than for older children (2 to ≤ 59 months). Agreement of RR classification into fast and normal breathing was moderate across all four devices, with Cohen's Kappa statistics ranging from 0.41 (SE 0.04) to 0.49 (SE 0.05).

INTERPRETATION

None of the four devices evaluated performed well based on agreement with the reference standard. The ARI timer currently recommended for use by CHWs should only be replaced by more expensive, equally performing, automated RR devices when aspects such as usability and duration of the device significantly improve the patient-provider experience.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].