Decision-support tools via mobile devices to improve quality of care in primary healthcare settings

Clinical decision support systems for maternity care: a systematic review and meta-analysis

Background

The use of Clinical Decision Support Systems (CDSS) is increasing throughout healthcare and may be able to improve safety and outcomes in maternity care, but maternity care has key differences to other disciplines that complicate the use of CDSS. We aimed to identify evaluated CDSS and synthesise evidence of their impact on maternity care.

Methods

We conducted a systematic review for articles published before 24th May 2024 that described i) CDSS that ii) investigated the impact of their use iii) in maternity settings. Medline, CINAHL, CENTRAL and HMIC were searched for articles relating to evaluations of CDSS in maternity settings, with forward- and backward-citation tracing conducted for included articles. Risk of bias was assessed using the Mixed Methods Assessment Tool, and CDSS were described according to the clinical problem, purpose, design, and technical environment. Quantitative results from articles reporting appropriate data were meta-analysed to estimate odds of a CDSS achieving its desired outcome using a multi-level random effects model, first by individual CDSS and then across all CDSS. PROSPERO ID: CRD42022348157.

Findings

We screened 12,039 papers and included 87 articles describing 47 unique CDSS. 24 articles (28%) described randomised controlled trials, 30 (34%) described non-randomised interventional studies, 10 (11%) described mixed methods studies, 10 (11%) described qualitative studies, 7 (8%) described quantitative descriptive studies, and 7 (8%) described economic evaluations. 49 (56%) were in High-Income Countries and 38 (44%) in Low- and Middle-Income countries, with no CDSS trialled in both income categories. Meta-analysis of 35 included studies found an odds ratio for improved outcomes of 1.69 (95% confidence interval 1.24-2.30). There was substantial variation in effects, aims, CDSS types, context, study designs, and outcomes.

Interpretation

Most CDSS evaluations showed improvements in outcomes, but there was heterogeneity in all aspects of design and evaluation of systems. CDSS are increasingly important in delivering healthcare, and Electronic Health Records and mHealth will increase their availability, but traditional epidemiological methods may be limited in guiding design and demonstrating effectiveness due to rapid CDSS development lifecycles and the complex systems in which they are embedded. Development methods that are attentive to context, such as Human Centred Design, will help to meet this need.

Funding

None.

Healthcare workers' informal uses of mobile phones and other mobile devices to support their work: a qualitative evidence synthesis

BACKGROUND

Healthcare workers sometimes develop their own informal solutions to deliver services. One such solution is to use their personal mobile phones or other mobile devices in ways that are unregulated by their workplace. This can help them carry out their work when their workplace lacks functional formal communication and information systems, but it can also lead to new challenges.

OBJECTIVES

To explore the views, experiences, and practices of healthcare workers, managers and other professionals working in healthcare services regarding their informal, innovative uses of mobile devices to support their work.

SEARCH METHODS

We searched MEDLINE, Embase, CINAHL and Scopus on 11 August 2022 for studies published since 2008 in any language. We carried out citation searches and contacted study authors to clarify published information and seek unpublished data.

SELECTION CRITERIA

We included qualitative studies and mixed-methods studies with a qualitative component. We included studies that explored healthcare workers' views, experiences, and practices regarding mobile phones and other mobile devices, and that included data about healthcare workers' informal use of these devices for work purposes.

DATA COLLECTION AND ANALYSIS

We extracted data using an extraction form designed for this synthesis, assessed methodological limitations using predefined criteria, and used a thematic synthesis approach to synthesise the data. We used the 'street-level bureaucrat' concept to apply a conceptual lens to our findings and prepare a line of argument that links these findings. We used the GRADE-CERQual approach to assess our confidence in the review findings and the line-of-argument statements. We collaborated with relevant stakeholders when defining the review scope, interpreting the findings, and developing implications for practice.

MAIN RESULTS

We included 30 studies in the review, published between 2013 and 2022. The studies were from high-, middle- and low-income countries and covered a range of healthcare settings and healthcare worker cadres. Most described mobile phone use as opposed to other mobile devices, such as tablets. We have moderate to high confidence in the statements in the following line of argument. The healthcare workers in this review, like other 'street-level bureaucrats', face a gap between what is expected of them and the resources available to them. To plug this gap, healthcare workers develop their own strategies, including using their own mobile phones, data and airtime. They also use other personal resources, including their personal time when taking and making calls outside working hours, and their personal networks when contacting others for help and advice. In some settings, healthcare workers' personal phone use, although unregulated, has become a normal part of many work processes. Some healthcare workers therefore experience pressure or expectations from colleagues and managers to use their personal phones. Some also feel driven to use their phones at work and at home because of feelings of obligation towards their patients and colleagues. At best, healthcare workers' use of their personal phones, time and networks helps humanise healthcare. It allows healthcare workers to be more flexible, efficient and responsive to the needs of the patient. It can give patients access to individual healthcare workers rather than generic systems and can help patients keep their sensitive information out of the formal system. It also allows healthcare workers to communicate with each other in more personalised, socially appropriate ways than formal systems allow. All of this can strengthen healthcare workers' relationships with community members and colleagues. However, these informal approaches can also replicate existing social hierarchies and deepen existing inequities among healthcare workers. Personal phone use costs healthcare workers money. This is a particular problem for lower-level healthcare workers and healthcare workers in low-income settings as they are likely to be paid less and may have less access to work phones or compensation. Out-of-hours use may also be more of a burden for lower-level healthcare workers, as they may find it harder to ignore calls when they are at home. Healthcare workers with poor access to electricity and the internet are less able to use informal mobile phone solutions, while healthcare workers who lack skills and training in how to appraise unendorsed online information are likely to struggle to identify trustworthy information. Informal digital channels can help healthcare workers expand their networks. But healthcare workers who rely on personal networks to seek help and advice are at a disadvantage if these networks are weak. Healthcare workers' use of their personal resources can also lead to problems for patients and can benefit some patients more than others. For instance, when healthcare workers store and share patient information on their personal phones, the confidentiality of this information may be broken. In addition, healthcare workers may decide to use their personal resources on some types of patients, but not others. Healthcare workers sometimes describe using their personal phones and their personal time and networks to help patients and clients whom they assess as being particularly in need. These decisions are likely to reflect their own values and ideas, for instance about social equity and patient 'worthiness'. But these may not necessarily reflect the goals, ideals and regulations of the formal healthcare system. Finally, informal mobile phone use plugs gaps in the system but can also weaken the system. The storing and sharing of information on personal phones and through informal channels can represent a 'shadow IT' (information technology) system where information about patient flow, logistics, etc., is not recorded in the formal system. Healthcare workers may also be more distracted at work, for instance, by calls from colleagues and family members or by social media use. Such challenges may be particularly difficult for weak healthcare systems.

AUTHORS' CONCLUSIONS

By finding their own informal solutions to workplace challenges, healthcare workers can be more efficient and more responsive to the needs of patients, colleagues and themselves. But these solutions also have several drawbacks. Efforts to strengthen formal health systems should consider how to retain the benefits of informal solutions and reduce their negative effects.

Community-based management of arterial hypertension and cardiovascular risk factors by lay village health workers for people with controlled and uncontrolled blood pressure in rural Lesotho: joint protocol for two cluster-randomized trials within the ComBaCaL cohort study (ComBaCaL aHT Twic 1 and ComBaCaL aHT TwiC 2)

BACKGROUND

Arterial hypertension (aHT) is a major cause for premature morbidity and mortality. Control rates remain poor, especially in low- and middle-income countries. Task-shifting to lay village health workers (VHWs) and the use of digital clinical decision support systems may help to overcome the current aHT care cascade gaps. However, evidence on the effectiveness of comprehensive VHW-led aHT care models, in which VHWs provide antihypertensive drug treatment and manage cardiovascular risk factors is scarce.

METHODS

Using the trials within the cohort (TwiCs) design, we are assessing the effectiveness of VHW-led aHT and cardiovascular risk management in two 1:1 cluster-randomized trials nested within the Community-Based chronic disease Care Lesotho (ComBaCaL) cohort study (NCT05596773). The ComBaCaL cohort study is maintained by trained VHWs and includes the consenting inhabitants of 103 randomly selected villages in rural Lesotho. After community-based aHT screening, adult, non-pregnant ComBaCaL cohort participants with uncontrolled aHT (blood pressure (BP) ≥ 140/90 mmHg) are enrolled in the aHT TwiC 1 and those with controlled aHT (BP < 140/90 mmHg) in the aHT TwiC 2. In intervention villages, VHWs offer lifestyle counseling, basic guideline-directed antihypertensive, lipid-lowering, and antiplatelet treatment supported by a tablet-based decision support application to eligible participants. In control villages, participants are referred to a health facility for therapeutic management. The primary endpoint for both TwiCs is the proportion of participants with controlled BP levels (< 140/90 mmHg) 12 months after enrolment. We hypothesize that the intervention is superior regarding BP control rates in participants with uncontrolled BP (aHT TwiC 1) and non-inferior in participants with controlled BP at baseline (aHT TwiC 2).

DISCUSSION

The TwiCs were launched on September 08, 2023. On May 20, 2024, 697 and 750 participants were enrolled in TwiC 1 and TwiC 2. To our knowledge, these TwiCs are the first trials to assess task-shifting of aHT care to VHWs at the community level, including the prescription of basic antihypertensive, lipid-lowering, and antiplatelet medication in Africa. The ComBaCaL cohort and nested TwiCs are operating within the routine VHW program and countries with similar community health worker programs may benefit from the findings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05684055. Registered on January 04, 2023.

Psychological interventions for depression in children and adolescents: A bibliometric analysis

BACKGROUND

Depression has gradually become a common psychological disorder among children and adolescents. Depression in children and adolescents affects their physical and mental development. Psychotherapy is considered to be one of the main treatment options for depressed children and adolescents. However, our understanding of the global performance and progress of psychological interventions for depression in children and adolescents (PIDCA) research is limited.

AIM

To identify collaborative research networks in this field and explore the current research status and hotspots through bibliometrics.

METHODS

Articles and reviews related to PIDCA from January 2010 to April 2023 were identified from the Web of Science Core Collection database. The Charticulator website, CiteSpace and VOSviewer software were used to visualize the trends in publications and citations, the collaborative research networks (countries, institutions, and authors), and the current research status and hotspots.

RESULTS

Until April 16, 2023, 1482 publications were identified. The number of documents published each year and citations had increased rapidly in this field. The United States had the highest productivity in this field. The most prolific institution was the University of London. Pim Cuijpers was the most prolific author. In the context of research related to PIDCA, both reference co-citation analysis and keywords co-occurrence analysis identified 10 research hotspots, including third-wave cognitive behavior therapy, short-term psychoanalytic psychotherapy, cognitive behavioral analysis system of psychotherapy, family element in psychotherapy, modular treatment, mobile-health, emotion-regulation-based transdiagnostic intervention program, dementia risk in later life, predictors of the efficacy of psychological intervention, and risks of psychological intervention.

CONCLUSION

This bibliometric study provides a comprehensive overview of PIDCA from 2010 to present. Psychological intervention characterized as psychological-process-focused, short, family-involved, modular, internet-based, emotion-regulation-based, and personalized may benefit more young people.

A Nationwide Chronic Disease Management Solution via Clinical Decision Support Services: Software Development and Real-Life Implementation Report

BACKGROUND

The increasing population of older adults has led to a rise in the demand for health care services, with chronic diseases being a major burden. Person-centered integrated care is required to address these challenges; hence, the Turkish Ministry of Health has initiated strategies to implement an integrated health care model for chronic disease management. We aim to present the design, development, nationwide implementation, and initial performance results of the national Disease Management Platform (DMP).

OBJECTIVE

This paper's objective is to present the design decisions taken and technical solutions provided to ensure successful nationwide implementation by addressing several challenges, including interoperability with existing IT systems, integration with clinical workflow, enabling transition of care, ease of use by health care professionals, scalability, high performance, and adaptability.

METHODS

The DMP is implemented as an integrated care solution that heavily uses clinical decision support services to coordinate effective screening and management of chronic diseases in adherence to evidence-based clinical guidelines and, hence, to increase the quality of health care delivery. The DMP is designed and implemented to be easily integrated with the existing regional and national health IT systems via conformance to international health IT standards, such as Health Level Seven Fast Healthcare Interoperability Resources. A repeatable cocreation strategy has been used to design and develop new disease modules to ensure extensibility while ensuring ease of use and seamless integration into the regular clinical workflow during patient encounters. The DMP is horizontally scalable in case of high load to ensure high performance.

RESULTS

As of September 2023, the DMP has been used by 25,568 health professionals to perform 73,715,269 encounters for 16,058,904 unique citizens. It has been used to screen and monitor chronic diseases such as obesity, cardiovascular risk, diabetes, and hypertension, resulting in the diagnosis of 3,545,573 patients with obesity, 534,423 patients with high cardiovascular risk, 490,346 patients with diabetes, and 144,768 patients with hypertension.

CONCLUSIONS

It has been demonstrated that the platform can scale horizontally and efficiently provides services to thousands of family medicine practitioners without performance problems. The system seamlessly interoperates with existing health IT solutions and runs as a part of the clinical workflow of physicians at the point of care. By automatically accessing and processing patient data from various sources to provide personalized care plan guidance, it maximizes the effect of evidence-based decision support services by seamless integration with point-of-care electronic health record systems. As the system is built on international code systems and standards, adaptation and deployment to additional regional and national settings become easily possible. The nationwide DMP as an integrated care solution has been operational since January 2020, coordinating effective screening and management of chronic diseases in adherence to evidence-based clinical guidelines.

A digital health algorithm to guide antibiotic prescription in pediatric outpatient care: a cluster randomized controlled trial

Excessive antibiotic use and antimicrobial resistance are major global public health threats. We developed ePOCT+, a digital clinical decision support algorithm in combination with C-reactive protein test, hemoglobin test, pulse oximeter and mentorship, to guide health-care providers in managing acutely sick children under 15 years old. To evaluate the impact of ePOCT+ compared to usual care, we conducted a cluster randomized controlled trial in Tanzanian primary care facilities. Over 11 months, 23,593 consultations were included from 20 ePOCT+ health facilities and 20,713 from 20 usual care facilities. The use of ePOCT+ in intervention facilities resulted in a reduction in the coprimary outcome of antibiotic prescription compared to usual care (23.2% versus 70.1%, adjusted difference -46.4%, 95% confidence interval (CI) -57.6 to -35.2). The coprimary outcome of day 7 clinical failure was noninferior in ePOCT+ facilities compared to usual care facilities (adjusted relative risk 0.97, 95% CI 0.85 to 1.10). There was no difference in the secondary safety outcomes of death and nonreferred secondary hospitalizations by day 7. Using ePOCT+ could help address the urgent problem of antimicrobial resistance by safely reducing antibiotic prescribing. Clinicaltrials.gov Identifier: NCT05144763.

Development and Pilot Implementation of Neotree, a Digital Quality Improvement Tool Designed to Improve Newborn Care and Survival in 3 Hospitals in Malawi and Zimbabwe: Cost Analysis Study

Background

Two-thirds of the 2.4 million newborn deaths that occurred in 2020 within the first 28 days of life might have been avoided by implementing existing low-cost evidence-based interventions for all sick and small newborns. An open-source digital quality improvement tool (Neotree) combining data capture with education and clinical decision support is a promising solution for this implementation gap.

Objective

We present results from a cost analysis of a pilot implementation of Neotree in 3 hospitals in Malawi and Zimbabwe.

Methods

We combined activity-based costing and expenditure approaches to estimate the development and implementation cost of a Neotree pilot in 1 hospital in Malawi, Kamuzu Central Hospital (KCH), and 2 hospitals in Zimbabwe, Sally Mugabe Central Hospital (SMCH) and Chinhoyi Provincial Hospital (CPH). We estimated the costs from a provider perspective over 12 months. Data were collected through expenditure reports, monthly staff time-use surveys, and project staff interviews. Sensitivity and scenario analyses were conducted to assess the impact of uncertainties on the results or estimate potential costs at scale. A pilot time-motion survey was conducted at KCH and a comparable hospital where Neotree was not implemented.

Results

Total cost of pilot implementation of Neotree at KCH, SMCH, and CPH was US $37,748, US $52,331, and US $41,764, respectively. Average monthly cost per admitted child was US $15, US $15, and US $58, respectively. Staff costs were the main cost component (average 73% of total costs, ranging from 63% to 79%). The results from the sensitivity analysis showed that uncertainty around the number of admissions had a significant impact on the costs in all hospitals. In Malawi, replacing monthly web hosting with a server also had a significant impact on the costs. Under routine (nonresearch) conditions and at scale, total costs are estimated to fall substantially, up to 76%, reducing cost per admitted child to as low as US $5 in KCH, US $4 in SMCH, and US $14 in CPH. Median time to admit a baby was 27 (IQR 20-40) minutes using Neotree (n=250) compared to 26 (IQR 21-30) minutes using paper-based systems (n=34), and the median time to discharge a baby was 9 (IQR 7-13) minutes for Neotree (n=246) compared to 3 (IQR 2-4) minutes for paper-based systems (n=50).

Conclusions

Neotree is a time- and cost-efficient tool, comparable with the results from limited similar mHealth decision-support tools in low- and middle-income countries. Implementation costs of Neotree varied substantially between the hospitals, mainly due to hospital size. The implementation costs could be substantially reduced at scale due to economies of scale because of integration to the health systems and reductions in cost items such as staff and overhead. More studies assessing the impact and cost-effectiveness of large-scale mHealth decision-support tools are needed.

Use of a digital job-aid in improving antenatal clinical protocols and quality of care in rural primary-level health facilities in Burkina Faso: a quasi-experimental evaluation

OBJECTIVE

We assessed the impact of a digital clinical decision support (CDS) tool in improving health providers adherence to recommended antenatal protocols and service quality in rural primary-level health facilities in Burkina Faso.

DESIGN

A quasi-experimental evaluation based on a cross-sectional post-intervention assessment comparing the intervention district to a comparison group.

SETTING AND PARTICIPANTS

The study included 331 direct observations and exit interviews of pregnant women seeking antenatal care (ANC) across 48 rural primary-level health facilities in Burkina Faso in 2021.

INTERVENTION

Digital CDS tool to improve health providers adherence to recommended antenatal protocols.

OUTCOME MEASURES

We analysed the quality of care on both the supply and demand sides. Quality-of-care service scores were based on actual care provided and expected care according to standards. Pregnant women's knowledge of counselling and satisfaction score after receiving care were also calculated. Other outcomes included time of clinical encounter.

RESULTS

The overall quality of health service provision was comparable across intervention and comparison health facilities (52% vs 51%) despite there being a significantly higher proportion of lower skilled providers in the intervention arm (42.5% vs 17.8%). On average, ANC visits were longer in the intervention area (median 24 min, IQR 18) versus comparison area (median 12 min, IQR: 8). The intervention arm had a significantly higher score difference in women's knowledge of received counselling (16.4 points, 95% CI 10.37 to 22.49), and women's satisfaction (16.18 points, 95% CI: 9.95 to 22.40).

CONCLUSION

Digital CDS tools provide a valuable opportunity to achieve substantial improvements of the quality of ANC and broadly maternal and newborn health in settings with high burden mortality and less trained health cadres when adequately implemented.

Digitalizing Clinical Guidelines: Experiences in the Development of Clinical Decision Support Algorithms for Management of Childhood Illness in Resource-Constrained Settings

Clinical decision support systems (CDSSs) can strengthen the quality of integrated management of childhood illness (IMCI) in resource-constrained settings. Several IMCI-related CDSSs have been developed and implemented in recent years. Yet, despite having a shared starting point, the IMCI-related CDSSs are markedly varied due to the need for interpretation when translating narrative guidelines into decision logic combined with considerations of context and design choices. Between October 2019 and April 2021, we conducted a comparative analysis of 4 IMCI-related CDSSs. The extent of adaptations to IMCI varied, but common themes emerged. Scope was extended to cover a broader range of conditions. Content was added or modified to enhance precision, align with new evidence, and support rational resource use. Structure was modified to increase efficiency, improve usability, and prioritize care for severely ill children. The multistakeholder development processes involved syntheses of recommendations from existing guidelines and literature; creation and validation of clinical algorithms; and iterative development, implementation, and evaluation. The common themes surrounding adaptations of IMCI guidance highlight the complexities of digitalizing evidence-based recommendations and reinforce the rationale for leveraging standards for CDSS development, such as the World Health Organization's SMART Guidelines. Implementation through multistakeholder dialogue is critical to ensure CDSSs can effectively and equitably improve quality of care for children in resource-constrained settings.

Investigating the Potential for Clinical Decision Support in Sub-Saharan Africa With AFYA (Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania): Protocol for a Prospective, Observational Pilot Study

BACKGROUND

Low- and middle-income countries face difficulties in providing adequate health care. One of the reasons is a shortage of qualified health workers. Diagnostic decision support systems are designed to aid clinicians in their work and have the potential to mitigate pressure on health care systems.

OBJECTIVE

The Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania (AFYA) study will evaluate the potential of an English-language artificial intelligence-based prototype diagnostic decision support system for mid-level health care practitioners in a low- or middle-income setting.

METHODS

This is an observational, prospective clinical study conducted in a busy Tanzanian district hospital. In addition to usual care visits, study participants will consult a mid-level health care practitioner, who will use a prototype diagnostic decision support system, and a study physician. The accuracy and comprehensiveness of the differential diagnosis provided by the diagnostic decision support system will be evaluated against a gold-standard differential diagnosis provided by an expert panel.

RESULTS

Patient recruitment started in October 2021. Participants were recruited directly in the waiting room of the outpatient clinic at the hospital. Data collection will conclude in May 2022. Data analysis is planned to be finished by the end of June 2022. The results will be published in a peer-reviewed journal.

CONCLUSIONS

Most diagnostic decision support systems have been developed and evaluated in high-income countries, but there is great potential for these systems to improve the delivery of health care in low- and middle-income countries. The findings of this real-patient study will provide insights based on the performance and usability of a prototype diagnostic decision support system in low- or middle-income countries.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04958577; http://clinicaltrials.gov/ct2/show/NCT04958577.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/34298.

Decision-support tools via mobile devices to improve quality of care in primary healthcare settings

BACKGROUND

The ubiquity of mobile devices has made it possible for clinical decision-support systems (CDSS) to become available to healthcare providers on handheld devices at the point-of-care, including in low- and middle-income countries. The use of CDSS by providers can potentially improve adherence to treatment protocols and patient outcomes. However, the evidence on the effect of the use of CDSS on mobile devices needs to be synthesized. This review was carried out to support a World Health Organization (WHO) guideline that aimed to inform investments on the use of decision-support tools on digital devices to strengthen primary healthcare.

OBJECTIVES

To assess the effects of digital clinical decision-support systems (CDSS) accessible via mobile devices by primary healthcare providers in the context of primary care settings.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, Global Index Medicus, POPLINE, and two trial registries from 1 January 2000 to 9 October 2020. We conducted a grey literature search using mHealthevidence.org and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies.

SELECTION CRITERIA

Study design: we included randomized trials, including full-text studies, conference abstracts, and unpublished data irrespective of publication status or language of publication.  Types of participants: we included studies of all cadres of healthcare providers, including lay health workers and other individuals (administrative, managerial, and supervisory staff) involved in the delivery of primary healthcare services using clinical decision-support tools; and studies of clients or patients receiving care from primary healthcare providers using digital decision-support tools. Types of interventions: we included studies comparing digital CDSS accessible via mobile devices with non-digital CDSS or no intervention, in the context of primary care. CDSS could include clinical protocols, checklists, and other job-aids which supported risk prioritization of patients. Mobile devices included mobile phones of any type (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones. We excluded studies where digital CDSS were used on laptops or integrated with electronic medical records or other types of longitudinal tracking of clients.

DATA COLLECTION AND ANALYSIS

A machine learning classifier that gave each record a probability score of being a randomized trial screened all search results. Two review authors screened titles and abstracts of studies with more than 10% probability of being a randomized trial, and one review author screened those with less than 10% probability of being a randomized trial. We followed standard methodological procedures expected by Cochrane and the Effective Practice and Organisation of Care group. We used the GRADE approach to assess the certainty of the evidence for the most important outcomes.

MAIN RESULTS

Eight randomized trials across varying healthcare contexts in the USA,. India, China, Guatemala, Ghana, and Kenya, met our inclusion criteria. A range of healthcare providers (facility and community-based, formally trained, and lay workers) used digital CDSS. Care was provided for the management of specific conditions such as cardiovascular disease, gastrointestinal risk assessment, and maternal and child health. The certainty of evidence ranged from very low to moderate, and we often downgraded evidence for risk of bias and imprecision. We are uncertain of the effect of this intervention on providers' adherence to recommended practice due to the very low certainty evidence (2 studies, 185 participants). The effect of the intervention on patients' and clients' health behaviours such as smoking and treatment adherence is mixed, with substantial variation across outcomes for similar types of behaviour (2 studies, 2262 participants). The intervention probably makes little or no difference to smoking rates among people at risk of cardiovascular disease but probably increases other types of desired behaviour among patients, such as adherence to treatment. The effect of the intervention on patients'/clients' health status and well-being  is also mixed (5 studies, 69,767 participants). It probably makes little or no difference to some types of health outcomes, but we are uncertain about other health outcomes, including maternal and neonatal deaths, due to very low-certainty evidence. The intervention may slightly improve patient or client acceptability and satisfaction (1 study, 187 participants). We found no studies that reported the time between the presentation of an illness and appropriate management, provider acceptability or satisfaction, resource use, or unintended consequences.

AUTHORS' CONCLUSIONS

We are uncertain about the effectiveness of mobile phone-based decision-support tools on several outcomes, including adherence to recommended practice. None of the studies had a quality of care framework and focused only on specific health areas.   We need well-designed research that takes a systems lens to assess these issues.