The Tools for Integrated Management of Childhood Illness (TIMCI) study protocol: a multi-country mixed-method evaluation of pulse oximetry and clinical decision support algorithms

Effective and sustainable strategies are needed to address the burden of preventable deaths among children under-five in resource-constrained settings. The Tools for Integrated Management of Childhood Illness (TIMCI) project aims to support healthcare providers to identify and manage severe illness, whilst promoting resource stewardship, by introducing pulse oximetry and clinical decision support algorithms (CDSAs) to primary care facilities in India, Kenya, Senegal and Tanzania. Health impact is assessed through: a pragmatic parallel group, superiority cluster randomised controlled trial (RCT), with primary care facilities randomly allocated (1:1) in India to pulse oximetry or control, and (1:1:1) in Tanzania to pulse oximetry plus CDSA, pulse oximetry, or control; and through a quasi-experimental pre-post study in Kenya and Senegal. Devices are implemented with guidance and training, mentorship, and community engagement. Sociodemographic and clinical data are collected from caregivers and records of enrolled sick children aged 0-59 months at study facilities, with phone follow-up on Day 7 (and Day 28 in the RCT). The primary outcomes assessed for the RCT are severe complications (mortality and secondary hospitalisations) by Day 7 and primary hospitalisations (within 24 hours and with referral); and, for the pre-post study, referrals and antibiotic. Secondary outcomes on other aspects of health status, hypoxaemia, referral, follow-up and antimicrobial prescription are also evaluated. In all countries, embedded mixed-method studies further evaluate the effects of the intervention on care and care processes, implementation, cost and cost-effectiveness. Pilot and baseline studies started mid-2021, RCT and post-intervention mid-2022, with anticipated completion mid-2023 and first results late-2023. Study approval has been granted by all relevant institutional review boards, national and WHO ethical review committees. Findings will be shared with communities, healthcare providers, Ministries of Health and other local, national and international stakeholders to facilitate evidence-based decision-making on scale-up.Study registration: NCT04910750 and NCT05065320.

Usability of pulse oximeters used by community health and primary care workers as screening tools for severe illness in children under five in low resource settings: A cross-sectional study in Cambodia, Ethiopia, South Sudan, and Uganda

Timely recognition and referral of severely ill children is especially critical in low-resource health systems. Pulse-oximeters can improve health outcomes of children by detecting hypoxaemia, a severity indicator of the most common causes of death in children. Cost-effectiveness of pulse-oximeters has been proven in low-income settings. However, evidence on their usability in community health settings is scarce.This study explores the usability of pulse-oximeters for community health and primary care workers in Cambodia, Ethiopia, South Sudan, and Uganda. We collected observational data, through a nine-task checklist, and survey data, using a five-point Likert scale questionnaire, capturing three usability aspects (effectiveness, efficiency, and satisfaction) of single-probe fingertip and multi-probe handheld devices. Effectiveness was determined by checklist completion rates and task completion rates per checklist item. Efficiency was reported as proportion of successful assessments within three attempts. Standardized summated questionnaire scores (min = 0, max = 100) determined health worker's satisfaction. Influencing factors on effectiveness and satisfaction were explored through hypothesis tests between independent groups (device type, cadre of health worker, country). Checklist completion rate was 78.3% [CI 72.6-83.0]. Choosing probes according to child age showed the lowest task completion rate of 68.7% [CI 60.3%-76.0%]. In 95.6% [CI 92.7%-97.4%] of assessments a reading was obtained within three attempts. The median satisfaction score was 95.6 [IQR = 92.2-99.0]. Significantly higher checklist completion rates were observed with single-probe fingertip devices (p<0.001) and children 12-59 months (p<0.001). We found higher satisfaction scores in South Sudan (p<0.001) and satisfaction varied slightly between devices. From a usability perspective single-probe devices for all age groups should be prioritized for scaled implementation. Further research on easy to use and accurate devices for infants is much needed.

ePOCT+ and the medAL-suite: Development of an electronic clinical decision support algorithm and digital platform for pediatric outpatients in low- and middle-income countries

Electronic clinical decision support algorithms (CDSAs) have been developed to address high childhood mortality and inappropriate antibiotic prescription by helping clinicians adhere to guidelines. Previously identified challenges of CDSAs include their limited scope, usability, and outdated clinical content. To address these challenges we developed ePOCT+, a CDSA for the care of pediatric outpatients in low- and middle-income settings, and the medical algorithm suite (medAL-suite), a software for the creation and execution of CDSAs. Following the principles of digital development, we aim to describe the process and lessons learnt from the development of ePOCT+ and the medAL-suite. In particular, this work outlines the systematic integrative development process in the design and implementation of these tools required to meet the needs of clinicians to improve uptake and quality of care. We considered the feasibility, acceptability and reliability of clinical signs and symptoms, as well as the diagnostic and prognostic performance of predictors. To assure clinical validity, and appropriateness for the country of implementation the algorithm underwent numerous reviews by clinical experts and health authorities from the implementing countries. The digitalization process involved the creation of medAL-creator, a digital platform which allows clinicians without IT programming skills to easily create the algorithms, and medAL-reader the mobile health (mHealth) application used by clinicians during the consultation. Extensive feasibility tests were done with feedback from end-users of multiple countries to improve the clinical algorithm and medAL-reader software. We hope that the development framework used for developing ePOCT+ will help support the development of other CDSAs, and that the open-source medAL-suite will enable others to easily and independently implement them. Further clinical validation studies are underway in Tanzania, Rwanda, Kenya, Senegal, and India.

Knowledge, attitudes and practices on childhood TB among healthcare workers

BACKGROUND: Increasing childhood TB case detection requires the deployment of diagnostic services at peripheral healthcare level. Capacity and readiness of healthcare workers (HCWs) are key to the delivery of innovative approaches.METHODS: In 2019, HCWs from five district hospitals (DHs) and 20 primary healthcare centres (PHCs) in Cambodia, Cameroon, Cote d´Ivoire, Sierra Leone and Uganda completed a self-administered knowledge-attitudes-practices (KAP) questionnaire on childhood TB. We computed knowledge and attitudes as scores and identified HCW characteristics associated with knowledge scores using linear regression.RESULT: Of 636 eligible HCWs, 497 (78%) participated. Median knowledge scores per country ranged between 7.4 and 12.1 (/18). Median attitude scores ranged between 2.8 and 3.3 (/4). Between 13.3% and 34.4% of HCWs reported diagnosing childhood with (presumptive) TB few times a week. Practising at PHC level, being female, being involved in indirect TB care, having a non-permanent position, having no previous research experience and working in Cambodia, Cameroon, Cote d´Ivoire and Sierra Leone as compared to Uganda were associated with a lower knowledge score.CONCLUSION: HCWs had overall limited knowledge, favourable attitudes and little practice of childhood TB diagnosis. Increasing HCW awareness, capacity and skills, and improving access to effective diagnosis are urgently needed.

Performance of five pulse oximeters to detect hypoxaemia as an indicator of severe illness in children under five by frontline health workers in low resource settings - A prospective, multicentre, single-blinded, trial in Cambodia, Ethiopia, South Sudan, and Uganda

BACKGROUND

Low blood oxygen saturation (SpO₂), or hypoxaemia, is an indicator of severe illness in children. Pulse oximetry is a globally accepted, non-invasive method to identify hypoxaemia, but rarely available outside higher-level facilities in resource-constrained countries. This study aims to evaluate the performance of different types of pulse oximeters amongst frontline health workers in Cambodia, Ethiopia, South Sudan, and Uganda.

METHODS

Five pulse oximeters (POx) which passed laboratory testing, out of an initial 32 potential pulse oximeters, were evaluated by frontline health workers for performance, defined as agreement between the SpO₂ measurements of the test device and the reference standard. The study protocol is registered with the Australia New Zealand Clinical Trials Registry (Ref: ACTRrn12615000348550).

FINDINGS

Two finger-tip pulse oximeters (Contec and Devon), two handheld pulse oximeters (Lifebox and Utech), and one phone pulse oximeter (Masimo) passed the laboratory testing. They were evaluated for performance on 1,313 children under five years old by 207 frontline health workers between February and May 2015. Phone and handheld pulse oximeters had greater overall agreement with the reference standard (56%; 95% CI 0.52 - 0.60 to 68%; 95% CI 0.65 - 0.71) than the finger-tip POx (31%; 95% CI 0.26 to 0.36 and 47%; 95% CI 0.42 to 0.52). Fingertip POx performance was substantially lower in the 0-2 month olds; having just 17% and 25% agreement. The finger-tip devices more often underreported SpO₂ readings (mean difference -7.9%; 95%CI -8.6,-7.2 and -3.9%; 95%CI -4.4,-3.4), and therefore over diagnosed hypoxaemia in the children assessed.

INTERPRETATION

While the Masimo phone pulse oximeter performed best, all handheld POx with age-specific probes performed well in the hands of frontline health workers, further highlighting their suitability as a screening tool of severe illness. The poor performance of the fingertip POx suggests they should not be used in children under five by frontline health workers. It is essential that POx are performance tested on children in routine settings (in vivo), not only in laboratories or controlled settings (in vitro), before being introduced at scale.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].

Performance of Four Respiratory Rate Counters to Support Community Health Workers to Detect the Symptoms of Pneumonia in Children in Low Resource Settings: A Prospective, Multicentre, Hospital-Based, Single-Blinded, Comparative Trial

BACKGROUND

Pneumonia is one of the leading causes of death in children under-five globally. The current diagnostic criteria for pneumonia are based on increased respiratory rate (RR) or chest in-drawing in children with cough and/or difficulty breathing. Accurately counting RR is difficult for community health workers (CHWs). Current RR counting devices are frequently inadequate or unavailable. This study analysed the performance of improved RR timers for detection of pneumonia symptoms in low-resource settings.

METHODS

Four RR timers were evaluated on 454 children, aged from 0 to 59 months with cough and/or difficulty breathing, over three months, by CHWs in hospital settings in Cambodia, Ethiopia, South Sudan and Uganda. The devices were the Mark Two ARI timer (MK2 ARI), counting beads with ARI timer, Rrate Android phone and the Respirometer feature phone applications. Performance was evaluated for agreement with an automated RR reference standard (Masimo Root patient monitoring and connectivity platform with ISA CO₂ capnography). This study is registered with ANZCTR [ACTRN12615000348550].

FINDINGS

While most CHWs managed to achieve a RR count with the four devices, the agreement was low for all; the mean difference of RR measurements from the reference standard for the four devices ranged from 0.5 (95% C.I. - 2.2 to 1.2) for the respirometer to 5.5 (95% C.I. 3.2 to 7.8) for Rrate. Performance was consistently lower for young infants (0 to < 2 months) than for older children (2 to ≤ 59 months). Agreement of RR classification into fast and normal breathing was moderate across all four devices, with Cohen's Kappa statistics ranging from 0.41 (SE 0.04) to 0.49 (SE 0.05).

INTERPRETATION

None of the four devices evaluated performed well based on agreement with the reference standard. The ARI timer currently recommended for use by CHWs should only be replaced by more expensive, equally performing, automated RR devices when aspects such as usability and duration of the device significantly improve the patient-provider experience.

FUNDING

Bill & Melinda Gates Foundation [OPP1054367].

Performance, Acceptability, and Usability of Respiratory Rate Timers and Pulse Oximeters When Used by Frontline Health Workers to Detect Symptoms of Pneumonia in Sub-Saharan Africa and Southeast Asia: Protocol for a Two-Phase, Multisite, Mixed-Methods Trial

Background

Pneumonia is one of the leading causes of death in children aged under 5 years in both sub-Saharan Africa and Southeast Asia. The current diagnostic criterion for pneumonia is based on the increased respiratory rate (RR) in children with cough or difficulty breathing. Low oxygen saturation, measured using pulse oximeters, is indicative of severe pneumonia. Health workers often find it difficult to accurately count the number of breaths, and the current RR counting devices are often difficult to use or unavailable. Nonetheless, improved counting devices and low-cost pulse oximeters are now available on the market.

Objective

The objective of our study was to identify the most accurate, usable, and acceptable devices for the diagnosis of pneumonia symptoms by community health workers and first-level health facility workers or frontline health workers in resource-poor settings.

Methods

This was a multicenter, prospective, two-stage, observational study to assess the performance and usability or acceptability of 9 potential diagnostic devices when used to detect symptoms of pneumonia in the hands of frontline health workers. Notably, 188 possible devices were ranked and scored, tested for suitability in a laboratory, and 5 pulse oximeters and 4 RR timers were evaluated for usability and performance by frontline health workers in hospital, health facility, and community settings. The performance was evaluated against 2 references over 3 months in Cambodia, Ethiopia, South Sudan, and Uganda. Furthermore, acceptability and usability was subsequently evaluated using both qualitative and quantitative methodologies in routine practice, over 3 months, in the 4 countries.

Results

This project was funded in 2014, and data collection has been completed. Data analysis is currently under way, and the first results are expected to be submitted for publication in 2018.

Conclusions

This is the first large-scale evaluation of tools to detect symptoms of pneumonia at the community level. In addition, selecting an appropriate reference standard against which the devices were measured was challenging given the lack of existing standards and differences of opinions among experts. The findings from this study will help create a standardized and validated protocol for future studies and support further comparative testing of diagnostic devices in these settings.

Trial Registration

Australian New Zealand Clinical Trials Registry ACTRN12615000348550; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=367306&isReview=true (Archived by Website at http://www.webcitation.org/72OcvgBcf)

International Registered Report Identifier (IRRID)

RR1-10.2196/10191

Childhood pneumonia diagnostics: community health workers' and national stakeholders' differing perspectives of new and existing aids

Background: Pneumonia heavily contributes to global under-five mortality. Many countries use community case management to detect and treat childhood pneumonia. Community health workers (CHWs) have limited tools to help them assess signs of pneumonia. New respiratory rate (RR) counting devices and pulse oximeters are being considered for this purpose.

Objective: To explore perspectives of CHWs and national stakeholders regarding the potential usability and scalability of seven devices to aid community assessment of pneumonia signs.

Design: Pile sorting was conducted to rate the usability and scalability of 7 different RR counting aids and pulse oximeters amongst 16 groups of participants. Following each pile-sorting session, a focus group discussion (FGD) explored participants’ sorting rationale. Purposive sampling was used to select CHWs and national stakeholders with experience in childhood pneumonia and integrated community case management (iCCM) in Cambodia, Ethiopia, Uganda and South Sudan. Pile-sorting data were aggregated for countries and participant groups. FGDs were audio recorded and transcribed verbatim. Translated FGDs transcripts were coded in NVivo 10 and analysed using thematic content analysis. Comparative analysis was performed between countries and groups to identify thematic patterns.

Results: CHWs and national stakeholders across the four countries perceived the acute respiratory infection (ARI) timer and fingertip pulse oximeter as highly scalable and easy for CHWs to use. National stakeholders were less receptive to new technologies. CHWs placed greater priority on device acceptability to caregivers and children. Both groups felt that heavy reliance on electricity reduced potential scalability and usability in rural areas. Device simplicity, affordability and sustainability were universally valued.

Conclusions: CHWs and national stakeholders prioritise different device characteristics according to their specific focus of work. The views of all relevant stakeholders, including health workers, policy makers, children and parents, should be considered in future policy decisions, research and development regarding suitable pneumonia diagnostic aids for community use.

Burden of disease in adults admitted to hospital in a rural region of coastal Kenya: an analysis of data from linked clinical and demographic surveillance systems

Background

Estimates of the burden of disease in adults in sub-Saharan Africa largely rely on models of sparse data. We aimed to measure the burden of disease in adults living in a rural area of coastal Kenya with use of linked clinical and demographic surveillance data.

Methods

We used data from 18 712 adults admitted to Kilifi District Hospital (Kilifi, Kenya) between Jan 1, 2007, and Dec 31, 2012, linked to 790 635 person-years of observation within the Kilifi Health and Demographic Surveillance System, to establish the rates and major causes of admission to hospital. These data were also used to model disease-specific disability-adjusted life-years lost in the population. We used geographical mapping software to calculate admission rates stratified by distance from the hospital.

Findings

The main causes of admission to hospital in women living within 5 km of the hospital were infectious and parasitic diseases (303 per 100 000 person-years of observation), pregnancy-related disorders (239 per 100 000 person-years of observation), and circulatory illnesses (105 per 100 000 person-years of observation). Leading causes of hospital admission in men living within 5 km of the hospital were infectious and parasitic diseases (169 per 100 000 person-years of observation), injuries (135 per 100 000 person-years of observation), and digestive system disorders (112 per 100 000 person-years of observation). HIV-related diseases were the leading cause of disability-adjusted life-years lost (2050 per 100 000 person-years of observation), followed by non-communicable diseases (741 per 100 000 person-years of observation). For every 5 km increase in distance from the hospital, all-cause admission rates decreased by 11% (95% CI 7–14) in men and 20% (17–23) in women. The magnitude of this decline was highest for endocrine disorders in women (35%; 95% CI 22–46) and neoplasms in men (30%; 9–45).

Interpretation

Adults in rural Kenya face a combined burden of infectious diseases, pregnancy-related disorders, cardiovascular illnesses, and injuries. Disease burden estimates based on hospital data are affected by distance from the hospital, and the amount of underestimation of disease burden differs by both disease and sex.

Funding

The Wellcome Trust, GAVI Alliance.

Validating physician-certified verbal autopsy and probabilistic modeling (InterVA) approaches to verbal autopsy interpretation using hospital causes of adult deaths

Background

The most common method for determining cause of death is certification by physicians based either on available medical records, or where such data are not available, through verbal autopsy (VA). The physician-certification approach is costly and inconvenient; however, recent work shows the potential of a computer-based probabilistic model (InterVA) to interpret verbal autopsy data in a more convenient, consistent, and rapid way. In this study we validate separately both physician-certified verbal autopsy (PCVA) and the InterVA probabilistic model against hospital cause of death (HCOD) in adults dying in a district hospital on the coast of Kenya.

Methods

Between March 2007 and June 2010, VA interviews were conducted for 145 adult deaths that occurred at Kilifi District Hospital. The VA data were reviewed by a physician and the cause of death established. A range of indicators (including age, gender, physical signs and symptoms, pregnancy status, medical history, and the circumstances of death) from the VA forms were included in the InterVA for interpretation. Cause-specific mortality fractions (CSMF), Cohen's kappa (κ) statistic, receiver operating characteristic (ROC) curves, sensitivity, specificity, and positive predictive values were applied to compare agreement between PCVA, InterVA, and HCOD.

Results

HCOD, InterVA, and PCVA yielded the same top five underlying causes of adult deaths. The InterVA overestimated tuberculosis as a cause of death compared to the HCOD. On the other hand, PCVA overestimated diabetes. Overall, CSMF for the five major cause groups by the InterVA, PCVA, and HCOD were 70%, 65%, and 60%, respectively. PCVA versus HCOD yielded a higher kappa value (κ = 0.52, 95% confidence interval [CI]: 0.48, 0.54) than the InterVA versus HCOD which yielded a kappa (κ) value of 0.32 (95% CI: 0.30, 0.38). Overall, (κ) agreement across the three methods was 0.41 (95% CI: 0.37, 0.48). The areas under the ROC curves were 0.82 for InterVA and 0.88 for PCVA. The observed sensitivities and specificities across the five major causes of death varied from 43% to 100% and 87% to 99%, respectively, for the InterVA/PCVA against the HCOD.

Conclusion

Both the InterVA and PCVA compared well with the HCOD at a population level and determined the top five underlying causes of death in the rural community of Kilifi. We hope that our study, albeit small, provides new and useful data that will stimulate further definitive work on methods of interpreting VA data.