1
|
Gannon H, Larsson L, Chimhuya S, Mangiza M, Wilson E, Kesler E, Chimhini G, Fitzgerald F, Zailani G, Crehan C, Khan N, Hull-Bailey T, Sassoon Y, Baradza M, Heys M, Chiume M. Development and Implementation of Digital Diagnostic Algorithms for Neonatal Units in Zimbabwe and Malawi: Development and Usability Study. JMIR Form Res 2024; 8:e54274. [PMID: 38277198 PMCID: PMC10858425 DOI: 10.2196/54274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 12/19/2023] [Accepted: 12/20/2023] [Indexed: 01/27/2024] Open
Abstract
BACKGROUND Despite an increase in hospital-based deliveries, neonatal mortality remains high in low-resource settings. Due to limited laboratory diagnostics, there is significant reliance on clinical findings to inform diagnoses. Accurate, evidence-based identification and management of neonatal conditions could improve outcomes by standardizing care. This could be achieved through digital clinical decision support (CDS) tools. Neotree is a digital, quality improvement platform that incorporates CDS, aiming to improve neonatal care in low-resource health care facilities. Before this study, first-phase CDS development included developing and implementing neonatal resuscitation algorithms, creating initial versions of CDS to address a range of neonatal conditions, and a Delphi study to review key algorithms. OBJECTIVE This second-phase study aims to codevelop and implement neonatal digital CDS algorithms in Malawi and Zimbabwe. METHODS Overall, 11 diagnosis-specific web-based workshops with Zimbabwean, Malawian, and UK neonatal experts were conducted (August 2021 to April 2022) encompassing the following: (1) review of available evidence, (2) review of country-specific guidelines (Essential Medicines List and Standard Treatment Guidelinesfor Zimbabwe and Care of the Infant and Newborn, Malawi), and (3) identification of uncertainties within the literature for future studies. After agreement of clinical content, the algorithms were programmed into a test script, tested with the respective hospital's health care professionals (HCPs), and refined according to their feedback. Once finalized, the algorithms were programmed into the Neotree software and implemented at the tertiary-level implementation sites: Sally Mugabe Central Hospital in Zimbabwe and Kamuzu Central Hospital in Malawi, in December 2021 and May 2022, respectively. In Zimbabwe, usability was evaluated through 2 usability workshops and usability questionnaires: Post-Study System Usability Questionnaire (PSSUQ) and System Usability Scale (SUS). RESULTS Overall, 11 evidence-based diagnostic and management algorithms were tailored to local resource availability. These refined algorithms were then integrated into Neotree. Where national management guidelines differed, country-specific guidelines were created. In total, 9 HCPs attended the usability workshops and completed the SUS, among whom 8 (89%) completed the PSSUQ. Both usability scores (SUS mean score 75.8 out of 100 [higher score is better]; PSSUQ overall score 2.28 out of 7 [lower score is better]) demonstrated high usability of the CDS function but highlighted issues around technical complexity, which continue to be addressed iteratively. CONCLUSIONS This study describes the successful development and implementation of the only known neonatal CDS system, incorporated within a bedside data capture system with the ability to deliver up-to-date management guidelines, tailored to local resource availability. This study highlighted the importance of collaborative participatory design. Further implementation evaluation is planned to guide and inform the development of health system and program strategies to support newborn HCPs, with the ultimate goal of reducing preventable neonatal morbidity and mortality in low-resource settings.
Collapse
Affiliation(s)
- Hannah Gannon
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
- Biomedical Research and Training Institute, Harare, Zimbabwe
| | - Leyla Larsson
- Institute of Computational Biology, Computational Health Centre, Helmholtz, Munich, Germany
| | - Simbarashe Chimhuya
- Department of Child, Adolescent and Women's Health, Faculty of Medicine and Health Science, University of Zimbabwe, Harare, Zimbabwe
| | | | - Emma Wilson
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
| | - Erin Kesler
- Children's Hospital of Philadelphia, Philidephia, PA, United States
| | - Gwendoline Chimhini
- Department of Child, Adolescent and Women's Health, Faculty of Medicine and Health Science, University of Zimbabwe, Harare, Zimbabwe
| | - Felicity Fitzgerald
- Biomedical Research and Training Institute, Harare, Zimbabwe
- Department of Infectious Disease, Imperial College London, London, United Kingdom
| | | | - Caroline Crehan
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
| | - Nushrat Khan
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
| | - Tim Hull-Bailey
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
| | | | | | - Michelle Heys
- Population, Policy and Practice, Institute of Child Health, University College London, London, United Kingdom
| | | |
Collapse
|
2
|
Haghparast-Bidgoli H, Hull-Bailey T, Nkhoma D, Chiyaka T, Wilson E, Fitzgerald F, Chimhini G, Khan N, Gannon H, Batura R, Cortina-Borja M, Larsson L, Chiume M, Sassoon Y, Chimhuya S, Heys M. Development and Pilot Implementation of Neotree, a Digital Quality Improvement Tool Designed to Improve Newborn Care and Survival in 3 Hospitals in Malawi and Zimbabwe: Cost Analysis Study. JMIR Mhealth Uhealth 2023; 11:e50467. [PMID: 38153802 PMCID: PMC10766148 DOI: 10.2196/50467] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 10/21/2023] [Accepted: 11/07/2023] [Indexed: 12/30/2023] Open
Abstract
Background Two-thirds of the 2.4 million newborn deaths that occurred in 2020 within the first 28 days of life might have been avoided by implementing existing low-cost evidence-based interventions for all sick and small newborns. An open-source digital quality improvement tool (Neotree) combining data capture with education and clinical decision support is a promising solution for this implementation gap. Objective We present results from a cost analysis of a pilot implementation of Neotree in 3 hospitals in Malawi and Zimbabwe. Methods We combined activity-based costing and expenditure approaches to estimate the development and implementation cost of a Neotree pilot in 1 hospital in Malawi, Kamuzu Central Hospital (KCH), and 2 hospitals in Zimbabwe, Sally Mugabe Central Hospital (SMCH) and Chinhoyi Provincial Hospital (CPH). We estimated the costs from a provider perspective over 12 months. Data were collected through expenditure reports, monthly staff time-use surveys, and project staff interviews. Sensitivity and scenario analyses were conducted to assess the impact of uncertainties on the results or estimate potential costs at scale. A pilot time-motion survey was conducted at KCH and a comparable hospital where Neotree was not implemented. Results Total cost of pilot implementation of Neotree at KCH, SMCH, and CPH was US $37,748, US $52,331, and US $41,764, respectively. Average monthly cost per admitted child was US $15, US $15, and US $58, respectively. Staff costs were the main cost component (average 73% of total costs, ranging from 63% to 79%). The results from the sensitivity analysis showed that uncertainty around the number of admissions had a significant impact on the costs in all hospitals. In Malawi, replacing monthly web hosting with a server also had a significant impact on the costs. Under routine (nonresearch) conditions and at scale, total costs are estimated to fall substantially, up to 76%, reducing cost per admitted child to as low as US $5 in KCH, US $4 in SMCH, and US $14 in CPH. Median time to admit a baby was 27 (IQR 20-40) minutes using Neotree (n=250) compared to 26 (IQR 21-30) minutes using paper-based systems (n=34), and the median time to discharge a baby was 9 (IQR 7-13) minutes for Neotree (n=246) compared to 3 (IQR 2-4) minutes for paper-based systems (n=50). Conclusions Neotree is a time- and cost-efficient tool, comparable with the results from limited similar mHealth decision-support tools in low- and middle-income countries. Implementation costs of Neotree varied substantially between the hospitals, mainly due to hospital size. The implementation costs could be substantially reduced at scale due to economies of scale because of integration to the health systems and reductions in cost items such as staff and overhead. More studies assessing the impact and cost-effectiveness of large-scale mHealth decision-support tools are needed.
Collapse
Affiliation(s)
| | - Tim Hull-Bailey
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | | | - Tarisai Chiyaka
- Centre for Sexual Health and HIV/AIDS Research, University of Zimbabwe, Harare, Zimbabwe
- Biomedical Research and Training Institute, Harare, Zimbabwe
| | - Emma Wilson
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Felicity Fitzgerald
- Department of Infectious Disease, Imperial College London, London, United Kingdom
| | - Gwendoline Chimhini
- Department of Child Adolescent and Women’s Health, University of Zimbabwe, Harare, Zimbabwe
| | - Nushrat Khan
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Hannah Gannon
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Rekha Batura
- Institute for Global Health, University College London, London, United Kingdom
| | - Mario Cortina-Borja
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Leyla Larsson
- Biomedical Research and Training Institute, Harare, Zimbabwe
| | | | | | - Simbarashe Chimhuya
- Department of Child Adolescent and Women’s Health, University of Zimbabwe, Harare, Zimbabwe
- Neonatal Unit, Sally Mugabe Central Hospital, Harare, Zimbabwe
| | - Michelle Heys
- Population, Policy and Practice Research and Teaching Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| |
Collapse
|
3
|
Heys M, Kesler E, Sassoon Y, Wilson E, Fitzgerald F, Gannon H, Hull-Bailey T, Chimhini G, Khan N, Cortina-Borja M, Nkhoma D, Chiyaka T, Stevenson A, Crehan C, Chiume ME, Chimhuya S. Development and implementation experience of a learning healthcare system for facility based newborn care in low resource settings: The Neotree. Learn Health Syst 2023; 7:e10310. [PMID: 36654803 PMCID: PMC9835040 DOI: 10.1002/lrh2.10310] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 02/28/2022] [Accepted: 03/20/2022] [Indexed: 01/21/2023] Open
Abstract
Introduction Improving peri- and postnatal facility-based care in low-resource settings (LRS) could save over 6000 babies' lives per day. Most of the annual 2.4 million neonatal deaths and 2 million stillbirths occur in healthcare facilities in LRS and are preventable through the implementation of cost-effective, simple, evidence-based interventions. However, their implementation is challenging in healthcare systems where one in four babies admitted to neonatal units die. In high-resource settings healthcare systems strengthening is increasingly delivered via learning healthcare systems to optimise care quality, but this approach is rare in LRS. Methods Since 2014 we have worked in Bangladesh, Malawi, Zimbabwe, and the UK to co-develop and pilot the Neotree system: an android application with accompanying data visualisation, linkage, and export. Its low-cost hardware and state-of-the-art software are used to support healthcare professionals to improve postnatal care at the bedside and to provide insights into population health trends. Here we summarise the formative conceptualisation, development, and preliminary implementation experience of the Neotree. Results Data thus far from ~18 000 babies, 400 healthcare professionals in four hospitals (two in Zimbabwe, two in Malawi) show high acceptability, feasibility, usability, and improvements in healthcare professionals' ability to deliver newborn care. The data also highlight gaps in knowledge in newborn care and quality improvement. Implementation has been resilient and informative during external crises, for example, coronavirus disease 2019 (COVID-19) pandemic. We have demonstrated evidence of improvements in clinical care and use of data for Quality Improvement (QI) projects. Conclusion Human-centred digital development of a QI system for newborn care has demonstrated the potential of a sustainable learning healthcare system to improve newborn care and outcomes in LRS. Pilot implementation evaluation is ongoing in three of the four aforementioned hospitals (two in Zimbabwe and one in Malawi) and a larger scale clinical cost effectiveness trial is planned.
Collapse
Affiliation(s)
- Michelle Heys
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Erin Kesler
- Children's Hospital of Philadelphia General, Thoracic, and Fetal Surgery Newborn Intensive Care Unit Philadelphia USA
| | | | - Emma Wilson
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Felicity Fitzgerald
- Infection, Immunity and Inflammation Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Hannah Gannon
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Tim Hull-Bailey
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Gwendoline Chimhini
- Department of Primary Healthcare Sciences University of Zimbabwe Harare Zimbabwe
| | - Nushrat Khan
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | - Mario Cortina-Borja
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | | | | | - Alex Stevenson
- Department of Primary Healthcare Sciences University of Zimbabwe Harare Zimbabwe.,Mbuya Nehanda Maternity Hospital Harare Zimbabwe
| | - Caroline Crehan
- Population, Policy and Practice Research and Teaching Department University College London Great Ormond Street Institute of Child Health London UK
| | | | - Simbarashe Chimhuya
- Department of Primary Healthcare Sciences University of Zimbabwe Harare Zimbabwe.,Maternity Division Sally Mugabe Central Hospital Harare Zimbabwe
| | | |
Collapse
|
4
|
Khan N, Crehan C, Hull-Bailey T, Normand C, Larsson L, Nkhoma D, Chiyaka T, Fitzgerald F, Kesler E, Gannon H, Kostkova P, Wilson E, Giaccone M, Krige D, Baradza M, Silksmith D, Neal S, Chimhuya S, Chiume M, Sassoon Y, Heys M. Software development process of Neotree - a data capture and decision support system to improve newborn healthcare in low-resource settings. Wellcome Open Res 2022; 7:305. [PMID: 38022734 PMCID: PMC10682609 DOI: 10.12688/wellcomeopenres.18423.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/30/2022] [Indexed: 12/01/2023] Open
Abstract
The global priority of improving neonatal survival could be tackled through the universal implementation of cost-effective maternal and newborn health interventions. Despite 90% of neonatal deaths occurring in low-resource settings, very few evidence-based digital health interventions exist to assist healthcare professionals in clinical decision-making in these settings. To bridge this gap, Neotree was co-developed through an iterative, user-centered design approach in collaboration with healthcare professionals in the UK, Bangladesh, Malawi, and Zimbabwe. It addresses a broad range of neonatal clinical diagnoses and healthcare indicators as opposed to being limited to specific conditions and follows national and international guidelines for newborn care. This digital health intervention includes a mobile application (app) which is designed to be used by healthcare professionals at the bedside. The app enables real-time data capture and provides education in newborn care and clinical decision support via integrated clinical management algorithms. Comprehensive routine patient data are prospectively collected regarding each newborn, as well as maternal data and blood test results, which are used to inform clinical decision making at the bedside. Data dashboards provide healthcare professionals and hospital management a near real-time overview of patient statistics that can be used for healthcare quality improvement purposes. To enable this workflow, the Neotree web editor allows fine-grained customization of the mobile app. The data pipeline manages data flow from the app to secure databases and then to the dashboard. Implemented in three hospitals in two countries so far, Neotree has captured routine data and supported the care of over 21,000 babies and has been used by over 450 healthcare professionals. All code and documentation are open source, allowing adoption and adaptation by clinicians, researchers, and developers.
Collapse
Affiliation(s)
- Nushrat Khan
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| | - Caroline Crehan
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| | | | | | - Leyla Larsson
- Biomedical Research and Training Institute (BRTI), Harare, Zimbabwe
| | | | - Tarisai Chiyaka
- Biomedical Research and Training Institute (BRTI), Harare, Zimbabwe
| | | | - Erin Kesler
- Children's Hospital of Philadelphia, Philadelphia, USA
| | - Hannah Gannon
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| | - Patty Kostkova
- UCL Centre for Digital Public Health in Emergencies, London, UK
| | - Emma Wilson
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| | | | - Danie Krige
- Baobab Web Services, City of Cape Town, South Africa
| | | | | | - Samuel Neal
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| | | | | | | | - Michelle Heys
- Population, Policy and Practice, UCL Great Ormond Street Institute of Child Health, London, WC1N 1EH, UK
| |
Collapse
|
5
|
Wilson E, Gannon H, Chimhini G, Fitzgerald F, Khan N, Lorencatto F, Kesler E, Nkhoma D, Chiyaka T, Haghparast-Bidgoli H, Lakhanpaul M, Cortina Borja M, Stevenson AG, Crehan C, Sassoon Y, Hull-Bailey T, Curtis K, Chiume M, Chimhuya S, Heys M. Protocol for an intervention development and pilot implementation evaluation study of an e-health solution to improve newborn care quality and survival in two low-resource settings, Malawi and Zimbabwe: Neotree. BMJ Open 2022; 12:e056605. [PMID: 35790332 PMCID: PMC9258512 DOI: 10.1136/bmjopen-2021-056605] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
INTRODUCTION Every year 2.4 million deaths occur worldwide in babies younger than 28 days. Approximately 70% of these deaths occur in low-resource settings because of failure to implement evidence-based interventions. Digital health technologies may offer an implementation solution. Since 2014, we have worked in Bangladesh, Malawi, Zimbabwe and the UK to develop and pilot Neotree: an android app with accompanying data visualisation, linkage and export. Its low-cost hardware and state-of-the-art software are used to improve bedside postnatal care and to provide insights into population health trends, to impact wider policy and practice. METHODS AND ANALYSIS This is a mixed methods (1) intervention codevelopment and optimisation and (2) pilot implementation evaluation (including economic evaluation) study. Neotree will be implemented in two hospitals in Zimbabwe, and one in Malawi. Over the 2-year study period clinical and demographic newborn data will be collected via Neotree, in addition to behavioural science informed qualitative and quantitative implementation evaluation and measures of cost, newborn care quality and usability. Neotree clinical decision support algorithms will be optimised according to best available evidence and clinical validation studies. ETHICS AND DISSEMINATION This is a Wellcome Trust funded project (215742_Z_19_Z). Research ethics approvals have been obtained: Malawi College of Medicine Research and Ethics Committee (P.01/20/2909; P.02/19/2613); UCL (17123/001, 6681/001, 5019/004); Medical Research Council Zimbabwe (MRCZ/A/2570), BRTI and JREC institutional review boards (AP155/2020; JREC/327/19), Sally Mugabe Hospital Ethics Committee (071119/64; 250418/48). Results will be disseminated via academic publications and public and policy engagement activities. In this study, the care for an estimated 15 000 babies across three sites will be impacted. TRIAL REGISTRATION NUMBER NCT0512707; Pre-results.
Collapse
Affiliation(s)
- Emma Wilson
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Hannah Gannon
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Gwendoline Chimhini
- Unit of Child and Adolescent Health, Faculty of Medicine and Health Science, University of Zimbabwe, Harare, Zimbabwe
| | - Felicity Fitzgerald
- Infection, Immunity and Inflammation Research & Teaching Department, UCL Great Ormond Street Institute of Child Health, London, London, UK
| | - Nushrat Khan
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | | | - Erin Kesler
- The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Deliwe Nkhoma
- Parent and Child Health Initiative Trust, Lilongwe, Central Region, Malawi
| | - Tarisai Chiyaka
- Biomedical Research and Training Institute, Harare, Zimbabwe
| | | | - Monica Lakhanpaul
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Mario Cortina Borja
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | | | - Caroline Crehan
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | | | - Tim Hull-Bailey
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| | | | - Msandeni Chiume
- Department of Paediatrics, Kamuzu Central Hospital, Lilongwe, Malawi
| | - Simbarashe Chimhuya
- Unit of Child and Adolescent Health, Faculty of Medicine and Health Sciences, University of Zimbabwe, Harare, Zimbabwe
| | - Michelle Heys
- Population, Policy and Practice Research and Teaching Department, UCL Great Ormond Street Institute of Child Health, London, UK
| |
Collapse
|
6
|
Crehan C, Chiume M, Mgusha Y, Dinga P, Hull-Bailey T, Normand C, Sassoon Y, Nkhoma D, Greenwood K, Lorencatto F, Lakhanpaul M, Heys M. Usability-Focused Development and Usage of NeoTree-Beta, an App for Newborn Care in a Low-Resource Neonatal Unit, Malawi. Front Public Health 2022; 10:793314. [PMID: 35570891 PMCID: PMC9096438 DOI: 10.3389/fpubh.2022.793314] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Accepted: 03/02/2022] [Indexed: 11/24/2022] Open
Abstract
Background Neonatal mortality is high in low-resource settings. NeoTree is a digital intervention for neonatal healthcare professionals (HCPs) aiming to achieve data-driven quality improvement and improved neonatal survival in low-resource hospitals. Optimising usability with end-users could help digital health interventions succeed beyond pilot stages in low-resource settings. Usability is the quality of a user's experience when interacting with an intervention, encompassing their effectiveness, efficiency, and overall satisfaction. Objective To evaluate the usability and usage of NeoTree beta-app and conduct Agile usability-focused intervention development. Method A real-world pilot of NeoTree beta-app was conducted over 6 months at Kamuzu Central Hospital neonatal unit, Malawi. Prior to deployment, think-aloud interviews were conducted to guide nurses through the app whilst voicing their thoughts aloud (n = 6). System Usability Scale (SUS) scores were collected before the implementation of NeoTree into usual clinical care and 6 months after implementation (n = 8 and 8). During the pilot, real-world user-feedback and user-data were gathered. Feedback notes were subjected to thematic analysis within an Agile “product backlog.” For usage, number of users, user-cadre, proportion of admissions/outcomes recorded digitally, and median app-completion times were calculated. Results Twelve overarching usability themes generated 57 app adjustments, 39 (68%) from think aloud analysis and 18 (32%) from the real-world testing. A total of 21 usability themes/issues with corresponding app features were produced and added to the app. Six themes relating to data collection included exhaustiveness of data schema, prevention of errors, ease of progression, efficiency of data entry using shortcuts, navigation of user interface (UI), and relevancy of content. Six themes relating to the clinical care included cohesion with ward process, embedded education, locally coherent language, adaptability of user-interface to available resources, and printout design to facilitate handover. SUS scores were above average (88.1 and 89.4 at 1 and 6 months, respectively). Ninety-three different HCPs of 5 cadres, recorded 1,323 admissions and 1,197 outcomes over 6 months. NeoTree achieved 100% digital coverage of sick neonates admitted. Median completion times were 16 and 8 min for admissions and outcomes, respectively. Conclusions This study demonstrates optimisation of a digital health app in a low-resource setting and could inform other similar usability studies apps in similar settings.
Collapse
Affiliation(s)
- Caroline Crehan
- Population Policy and Practice Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Msandeni Chiume
- Paediatric Department, Kamuzu Central Hospital, Lilongwe, Malawi
| | - Yamikani Mgusha
- Paediatric Department, Kamuzu Central Hospital, Lilongwe, Malawi
| | - Precious Dinga
- Paediatric Department, Kamuzu Central Hospital, Lilongwe, Malawi
| | - Tim Hull-Bailey
- Population Policy and Practice Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | | | | | - Deliwe Nkhoma
- Parent and Child Health Initiative, Lilongwe, Malawi
| | | | - Fabiana Lorencatto
- Centre for Behaviour Change, University College London, London, United Kingdom
| | - Monica Lakhanpaul
- Population Policy and Practice Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| | - Michelle Heys
- Population Policy and Practice Department, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom
| |
Collapse
|