Unveiling the Black Box of Diagnostic and Clinical Decision Support Systems for Antenatal Care: Realist Evaluation

Antenatal care practices for gestational weight gain: a cross sectional survey of antenatal care providers reported provision and barriers to providing recommended care

BACKGROUND

Implementation of recommended gestational weight gain (GWG) care by antenatal care providers is poor. It is unclear whether practice implementation and barriers differ between antenatal care provider profession or experience. This study aimed to assesses the provision of and barriers to guideline care for GWG and examine associations with professional discipline and years of experience.

METHODS

A cross sectional survey was conducted with antenatal care providers working in three public maternity services in a regional city in Australia. Data were collected on the provision of and barriers (informed by the Theoretical Domains Framework) to recommended GWG care. Data were summarised using descriptive statistics. Associations between health profession characteristics (professional discipline and years providing antenatal care) and GWG care practices and barrier outcomes were assessed using multivariate logistic regression.

RESULTS

117 antenatal care providers completed the survey (75% participation rate). One quarter (25%) reported that they routinely provided recommended GWG assessment at the first antenatal visit, and 9% at subsequent visits. Only 7% routinely provided recommended advice on GWG, healthy eating and physical activity. Professional discipline or years of experience were not associated with higher odds of GWG practices. Skills, belief about capabilities, belief about consequences and environmental context and resources were barriers to providing care. Medical professionals had higher odds of agreeing that they have been adequately trained to address GWG (OR = 9.14, 95%CI:3.10-26.90) and feel competent in having sensitive conversations with pregnant women about GWG (OR = 8.60, 95%CI:2.29-32.28) than midwives. Midwives had higher odds of agreeing that there are services they can refer pregnant women to for further support (OR = 2.80, 95%CI:1.13-6.91).

CONCLUSIONS

The provision of antenatal care for GWG was low, inconsistently provided and did not differ by professional discipline or years of experience. Antenatal care providers report numerous barriers including skills, belief about capabilities, belief about consequences, and environmental context and resources. Barriers to GWG care provision differed by professional discipline, but not years of providing care. The findings demonstrate that the type and prioritisation of practice-change implementation strategies may need to be tailored to address the differential barriers faced by professional groups.

Clinical decision support systems for maternity care: a systematic review and meta-analysis

Background

The use of Clinical Decision Support Systems (CDSS) is increasing throughout healthcare and may be able to improve safety and outcomes in maternity care, but maternity care has key differences to other disciplines that complicate the use of CDSS. We aimed to identify evaluated CDSS and synthesise evidence of their impact on maternity care.

Methods

We conducted a systematic review for articles published before 24th May 2024 that described i) CDSS that ii) investigated the impact of their use iii) in maternity settings. Medline, CINAHL, CENTRAL and HMIC were searched for articles relating to evaluations of CDSS in maternity settings, with forward- and backward-citation tracing conducted for included articles. Risk of bias was assessed using the Mixed Methods Assessment Tool, and CDSS were described according to the clinical problem, purpose, design, and technical environment. Quantitative results from articles reporting appropriate data were meta-analysed to estimate odds of a CDSS achieving its desired outcome using a multi-level random effects model, first by individual CDSS and then across all CDSS. PROSPERO ID: CRD42022348157.

Findings

We screened 12,039 papers and included 87 articles describing 47 unique CDSS. 24 articles (28%) described randomised controlled trials, 30 (34%) described non-randomised interventional studies, 10 (11%) described mixed methods studies, 10 (11%) described qualitative studies, 7 (8%) described quantitative descriptive studies, and 7 (8%) described economic evaluations. 49 (56%) were in High-Income Countries and 38 (44%) in Low- and Middle-Income countries, with no CDSS trialled in both income categories. Meta-analysis of 35 included studies found an odds ratio for improved outcomes of 1.69 (95% confidence interval 1.24-2.30). There was substantial variation in effects, aims, CDSS types, context, study designs, and outcomes.

Interpretation

Most CDSS evaluations showed improvements in outcomes, but there was heterogeneity in all aspects of design and evaluation of systems. CDSS are increasingly important in delivering healthcare, and Electronic Health Records and mHealth will increase their availability, but traditional epidemiological methods may be limited in guiding design and demonstrating effectiveness due to rapid CDSS development lifecycles and the complex systems in which they are embedded. Development methods that are attentive to context, such as Human Centred Design, will help to meet this need.

Funding

None.

Exploring the role of professional identity in the implementation of clinical decision support systems-a narrative review

BACKGROUND

Clinical decision support systems (CDSSs) have the potential to improve quality of care, patient safety, and efficiency because of their ability to perform medical tasks in a more data-driven, evidence-based, and semi-autonomous way. However, CDSSs may also affect the professional identity of health professionals. Some professionals might experience these systems as a threat to their professional identity, as CDSSs could partially substitute clinical competencies, autonomy, or control over the care process. Other professionals may experience an empowerment of the role in the medical system. The purpose of this study is to uncover the role of professional identity in CDSS implementation and to identify core human, technological, and organizational factors that may determine the effect of CDSSs on professional identity.

METHODS

We conducted a systematic literature review and included peer-reviewed empirical studies from two electronic databases (PubMed, Web of Science) that reported on key factors to CDSS implementation and were published between 2010 and 2023. Our explorative, inductive thematic analysis assessed the antecedents of professional identity-related mechanisms from the perspective of different health care professionals (i.e., physicians, residents, nurse practitioners, pharmacists).

RESULTS

One hundred thirty-one qualitative, quantitative, or mixed-method studies from over 60 journals were included in this review. The thematic analysis found three dimensions of professional identity-related mechanisms that influence CDSS implementation success: perceived threat or enhancement of professional control and autonomy, perceived threat or enhancement of professional skills and expertise, and perceived loss or gain of control over patient relationships. At the technological level, the most common issues were the system's ability to fit into existing clinical workflows and organizational structures, and its ability to meet user needs. At the organizational level, time pressure and tension, as well as internal communication and involvement of end users were most frequently reported. At the human level, individual attitudes and emotional responses, as well as familiarity with the system, most often influenced the CDSS implementation. Our results show that professional identity-related mechanisms are driven by these factors and influence CDSS implementation success. The perception of the change of professional identity is influenced by the user's professional status and expertise and is improved over the course of implementation.

CONCLUSION

This review highlights the need for health care managers to evaluate perceived professional identity threats to health care professionals across all implementation phases when introducing a CDSS and to consider their varying manifestations among different health care professionals. Moreover, it highlights the importance of innovation and change management approaches, such as involving health professionals in the design and implementation process to mitigate threat perceptions. We provide future areas of research for the evaluation of the professional identity construct within health care.

Longitudinal realist evaluation of the Dementia PersonAlised Care Team (D-PACT) intervention: protocol

BACKGROUND

Different dementia support roles exist but evidence is lacking on which aspects are best, for whom, and in what circumstances, and on their associated costs and benefits. Phase 1 of the Dementia PersonAlised Care Team programme (D-PACT) developed a post-diagnostic primary care-based intervention for people with dementia and their carers and assessed the feasibility of a trial.

AIM

Phase 2 of the programme aims to 1) refine the programme theory on how, when, and for whom the intervention works; and 2) evaluate its value and impact.

DESIGN & SETTING

A realist longitudinal mixed-methods evaluation will be conducted in urban, rural, and coastal areas across South West and North West England where low-income or ethnic minority populations (for example, South Asian) are represented. Design was informed by patient, public, and professional stakeholder input and phase 1 findings.

METHOD

High-volume qualitative and quantitative data will be collected longitudinally from people with dementia, carers, and practitioners. Analyses will comprise the following: 1) realist longitudinal case studies; 2) conversation analysis of recorded interactions; 3) statistical analyses of outcome and experience questionnaires; 4a) health economic analysis examining costs of delivery; and 4b) realist economic analysis of high-cost events and 'near misses'. All findings will be synthesised using a joint display table, evidence appraisal tool, triangulation, and stakeholder co-analysis.

CONCLUSION

The realist evaluation will describe how, why, and for whom the intervention does or does not lead to change over time. It will also demonstrate how a non-randomised design can be more appropriate for complex interventions with similar questions or populations.

Scoping review of health information technology usability methods leveraged in Africa

OBJECTIVE

The aim of this study was to explore the state of health information technology (HIT) usability evaluation in Africa.

MATERIALS AND METHODS

We searched three electronic databases: PubMed, Embase, and Association for Computing Machinery. We categorized the stage of evaluations, the type of interactions assessed, and methods applied using Stead's System Development Life Cycle (SDLC) and Bennett and Shackel's usability models.

RESULTS

Analysis of 73 of 1002 articles that met inclusion criteria reveals that HIT usability evaluations in Africa have increased in recent years and mainly focused on later SDLC stage (stages 4 and 5) evaluations in sub-Saharan Africa. Forty percent of the articles examined system-user-task-environment (type 4) interactions. Most articles used mixed methods to measure usability. Interviews and surveys were often used at each development stage, while other methods, such as quality-adjusted life year analysis, were only found at stage 5. Sixty percent of articles did not include a theoretical model or framework.

DISCUSSION

The use of multistage evaluation and mixed methods approaches to obtain a comprehensive understanding HIT usability is critical to ensure that HIT meets user needs.

CONCLUSIONS

Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. Early-stage evaluations (stages 1 and 2) and interactions (types 0 and 1) should receive special attention to ensure HIT usability prior to implementing HIT in the field.

How do participatory methods shape policy? Applying a realist approach to the formulation of a new tuberculosis policy in Georgia

OBJECTIVES

This paper presents the iterative process of participatory multistakeholder engagement that informed the development of a new national tuberculosis (TB) policy in Georgia, and the lessons learnt.

METHODS

Guided by realist evaluation methods, a multistakeholder dialogue was organised to elicit stakeholders' assumptions on challenges and possible solutions for better TB control. Two participatory workshops were conducted with key actors, interspersed by reflection meetings within the research team and discussions with policymakers. Using concept mapping and causal mapping techniques, and drawing causal loop diagrams, we visualised how actors understood TB service provision challenges and the potential means by which a results-based financing (RBF) policy could address these.

SETTING

The study was conducted in Tbilisi, Georgia.

PARTICIPANTS

A total of 64 key actors from the Ministry of Labour, Health and Social Affairs, staff of the Global Fund to Fight AIDS, TB and Malaria Georgia Project, the National Centre for Disease Control and Public Health, the National TB programme, TB service providers and members of the research team were involved in the workshops.

RESULTS

Findings showed that beyond provider incentives, additional policy components were necessary. These included broadening the incentive package to include institutional and organisational incentives, retraining service providers, clear redistribution of roles to support an integrated care model, and refinement of monitoring tools. Health system elements, such as effective referral systems and health information systems were highlighted as necessary for service improvement.

CONCLUSIONS

Developing policies that address complex issues requires methods that facilitate linkages between multiple stakeholders and between theory and practice. Such participatory approaches can be informed by realist evaluation principles and visually facilitated by causal loop diagrams. This approach allowed us leverage stakeholders' knowledge and expertise on TB service delivery and RBF to codesign a new policy.

Barriers and facilitators to implementing evidence-based health innovations in low- and middle-income countries: A systematic literature review

The unsuccessful implementation of health innovations occurs frequently, leading to missed opportunities where improvements could have been made on various aspects of a health system. The purpose of this study is to identify, assess and synthesise the facilitators and barriers to sustainably implementing evidence-based health innovations in a low- and middle-income country (LMIC) context. To identify the LMIC specific facilitators and barriers, a systematic literature review was conducted. 79 studies were analysed, and the implementation barriers and facilitators identified in each study were extracted. The extracted barriers and facilitators were categorised and synthesized into one of seven concepts: context, innovation, relations and networks, institutions, knowledge, actors or resources. There were no substantial variations between the frequency that each concept was identified as a facilitator or barrier to implementation. However, resources, which includes time, human, financial and physical resources, was the most frequently mentioned concept; emphasising the need to focus on the resource situation in LMICs. This study contributes to the growing literature that aims to inform health system planners and evaluators in LMICs on effectively and sustainably implementing evidence-based health innovations.

Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis

BACKGROUND

Mobile health (mHealth), refers to healthcare practices supported by mobile devices, such as mobile phones and tablets. Within primary care, health workers often use mobile devices to register clients, track their health, and make decisions about care, as well as to communicate with clients and other health workers. An understanding of how health workers relate to, and experience mHealth, can help in its implementation.

OBJECTIVES

To synthesise qualitative research evidence on health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services, and to develop hypotheses about why some technologies are more effective than others.

SEARCH METHODS

We searched MEDLINE, Embase, CINAHL, Science Citation Index and Social Sciences Citation Index in January 2018. We searched Global Health in December 2015. We screened the reference lists of included studies and key references and searched seven sources for grey literature (16 February to 5 March 2018). We re-ran the search strategies in February 2020. We screened these records and any studies that we identified as potentially relevant are awaiting classification.

SELECTION CRITERIA

We included studies that used qualitative data collection and analysis methods. We included studies of mHealth programmes that were part of primary healthcare services. These services could be implemented in public or private primary healthcare facilities, community and workplace, or the homes of clients. We included all categories of health workers, as well as those persons who supported the delivery and management of the mHealth programmes. We excluded participants identified as technical staff who developed and maintained the mHealth technology, without otherwise being involved in the programme delivery. We included studies conducted in any country.

DATA COLLECTION AND ANALYSIS

We assessed abstracts, titles and full-text papers according to the inclusion criteria. We found 53 studies that met the inclusion criteria and sampled 43 of these for our analysis. For the 43 sampled studies, we extracted information, such as country, health worker category, and the mHealth technology. We used a thematic analysis process. We used GRADE-CERQual to assess our confidence in the findings.

MAIN RESULTS

Most of the 43 included sample studies were from low- or middle-income countries. In many of the studies, the mobile devices had decision support software loaded onto them, which showed the steps the health workers had to follow when they provided health care. Other uses included in-person and/or text message communication, and recording clients' health information. Almost half of the studies looked at health workers' use of mobile devices for mother, child, and newborn health. We have moderate or high confidence in the following findings. mHealth changed how health workers worked with each other: health workers appreciated being more connected to colleagues, and thought that this improved co-ordination and quality of care. However, some described problems when senior colleagues did not respond or responded in anger. Some preferred face-to-face connection with colleagues. Some believed that mHealth improved their reporting, while others compared it to "big brother watching". mHealth changed how health workers delivered care: health workers appreciated how mHealth let them take on new tasks, work flexibly, and reach clients in difficult-to-reach areas. They appreciated mHealth when it improved feedback, speed and workflow, but not when it was slow or time consuming. Some health workers found decision support software useful; others thought it threatened their clinical skills. Most health workers saw mHealth as better than paper, but some preferred paper. Some health workers saw mHealth as creating more work. mHealth led to new forms of engagement and relationships with clients and communities: health workers felt that communicating with clients by mobile phone improved care and their relationships with clients, but felt that some clients needed face-to-face contact. Health workers were aware of the importance of protecting confidential client information when using mobile devices. Some health workers did not mind being contacted by clients outside working hours, while others wanted boundaries. Health workers described how some community members trusted health workers that used mHealth while others were sceptical. Health workers pointed to problems when clients needed to own their own phones. Health workers' use and perceptions of mHealth could be influenced by factors tied to costs, the health worker, the technology, the health system and society, poor network access, and poor access to electricity: some health workers did not mind covering extra costs. Others complained that phone credit was not delivered on time. Health workers who were accustomed to using mobile phones were sometimes more positive towards mHealth. Others with less experience, were sometimes embarrassed about making mistakes in front of clients or worried about job security. Health workers wanted training, technical support, user-friendly devices, and systems that were integrated into existing electronic health systems. The main challenges health workers experienced were poor network connections, access to electricity, and the cost of recharging phones. Other problems included damaged phones. Factors outside the health system also influenced how health workers experienced mHealth, including language, gender, and poverty issues. Health workers felt that their commitment to clients helped them cope with these challenges.

AUTHORS' CONCLUSIONS

Our findings propose a nuanced view about mHealth programmes. The complexities of healthcare delivery and human interactions defy simplistic conclusions on how health workers will perceive and experience their use of mHealth. Perceptions reflect the interplay between the technology, contexts, and human attributes. Detailed descriptions of the programme, implementation processes and contexts, alongside effectiveness studies, will help to unravel this interplay to formulate hypotheses regarding the effectiveness of mHealth.

Global Health Research, Anthropology and Realist Enquiry

In this article, I set out to capture the dynamics of two streams within the field of global health research: realist research and medical anthropology. I critically discuss the development of methodology and practice in realist health research in low- and middle-income countries against the background of anthropological practice in global health to make claims on why realist enquiry has taken a high flight. I argue that in order to provide a contribution to today’s complex global issues, we need to adopt a pragmatic stance and move past disciplinary silos: both methodologies have the potential to be well-suited to an analysis of deep layers of context and of key social mechanisms.