Developing a learning health system: Insights from a qualitative process evaluation of a pharmacist-led electronic audit and feedback intervention to improve medication safety in primary care

Collaborative discussions between GPs and pharmacists to optimise patient medication: a qualitative study within a UK primary care clinical trial

BACKGROUND

There has been significant investment in pharmacists working in UK general practice to improve the effective and safe use of medicines. However, evidence of how to optimise collaboration between GPs and pharmacists in the context of polypharmacy (multiple medication) is lacking.

AIM

To explore GP and pharmacist views and experiences of in-person, interprofessional collaborative discussions (IPCDs) as part of a complex intervention to optimise medication use for patients with polypharmacy in general practice.

DESIGN AND SETTING

A mixed-method process evaluation embedded within the Improving Medicines use in People with Polypharmacy in Primary Care (IMPPP) trial conducted in Bristol and the West Midlands, between February 2021 and September 2023.

METHOD

Audio-recordings of IPCDs between GPs and pharmacists, along with individual semi-structured interviews to explore their reflections on these discussions, were used. All recordings were transcribed verbatim and analysed thematically.

RESULTS

A total of 14 practices took part in the process evaluation from February 2022 to September 2023; 17 IPCD meetings were audio-recorded, discussing 30 patients (range 1-6 patients per meeting). In all, six GPs and 13 pharmacists were interviewed. The IPCD was highly valued by GPs and pharmacists who described benefits, including: strengthening their working relationship; gaining in confidence to manage more complex patients; and learning from each other. It was often challenging, however, to find time for the IPCDs.

CONCLUSION

The model of IPCD used in this study provided protected time for GPs and pharmacists to work together to deliver whole-patient care, with both professions finding this beneficial. Protected time for interprofessional liaison and collaboration, and structured interventions may facilitate improved patient care.

Development and application of Breadth-Depth-Context (BDC), a conceptual framework for measuring technology engagement with a qualified clinical data registry

Objectives

Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR.

Materials and Methods

We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles.

Results

We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors.

Discussion

We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement.

Conclusion

Going forward, the BDC framework can be used to study engagement with similar dashboards.

Pharmacist-led primary care interventions to promote medicines optimisation and reduce overprescribing: a systematic review of UK studies and initiatives

OBJECTIVES

To systematically review and synthesise evidence on the effectiveness and implementation barriers/facilitators of pharmacist-led interventions to promote medicines optimisation and reduce overprescribing in UK primary care.

DESIGN

Systematic review.

SETTING

UK primary care.

METHODS

We searched MEDLINE, Embase, CINAHL PsycINFO and The Cochrane Library for UK-based studies published between January 2013 and February 2023. Targeted searches for grey literature were conducted in May 2023. Quantitative and qualitative studies (including conference abstracts and grey literature) that addressed a relevant intervention and reported a primary outcome related to changes in prescribing were eligible for inclusion. Quality of included studies was assessed using the Multiple Methods Appraisal Tool. We performed a narrative synthesis, grouping studies by publication status, setting and type of data reported (effectiveness or implementation).

RESULTS

We included 14 peer-reviewed journal articles and 11 conference abstracts, together with 4 case study reports. The journal articles reported 10 different interventions, 5 delivered in general practice, 4 in care homes and 1 in community pharmacy. The quality of evidence was higher in general practice than in care home settings. It was consistently reported that the intervention improved outcomes related to prescribing, although the limited number of studies and wide range of outcomes reported made it difficult to estimate the size of any effect. Implementation was strongly influenced by relationships between pharmacists and other health and care professionals, especially general practitioners. Implementation in care homes appeared to be more complex than in general practice because of differences in systems and 'culture' between health and social care.

CONCLUSIONS

Pharmacist-led interventions have been reported to reduce overprescribing in primary care settings in the UK but a shortage of high-quality evidence means that more rigorous studies using high-quality designs are needed. More research is also needed in community pharmacy settings; to assess intervention effects on patient outcomes other than prescribing and to investigate how reducing overprescribing can impact health inequalities.

PROSPERO REGISTRATION NUMBER

CRD42023396366.

Learning health systems on the front lines to strengthen care against future pandemics and climate change: a rapid review

BACKGROUND

An essential component of future-proofing health systems against future pandemics and climate change is strengthening the front lines of care: principally, emergency departments and primary care settings. To achieve this, these settings can adopt learning health system (LHS) principles, integrating data, evidence, and experience to continuously improve care delivery. This rapid review aimed to understand the ways in which LHS principles have been applied to primary care and emergency departments, the extent to which LHS approaches have been adopted in these key settings, and the factors that affect their adoption.

METHODS

Three academic databases (Embase, Scopus, and PubMed) were searched for full text articles reporting on LHSs in primary care and/or emergency departments published in the last five years. Articles were included if they had a primary focus on LHSs in primary care settings (general practice, allied health, multidisciplinary primary care, and community-based care) and/or emergency care settings. Data from included articles were catalogued and synthesised according to the modified Institute of Medicine's five-component framework for LHSs (science and informatics, patient-clinician partnerships, incentives, continuous learning culture, and structure and governance).

RESULTS

Thirty-seven articles were included, 32 of which reported LHSs in primary care settings and seven of which reported LHSs in emergency departments. Science and informatics was the most commonly reported LHS component, followed closely by continuous learning culture and structure and governance. Most articles (n = 30) reported on LHSs that had been adopted, and many of the included articles (n = 17) were descriptive reports of LHS approaches.

CONCLUSIONS

Developing LHSs at the front lines of care is essential for future-proofing against current and new threats to health system sustainability, such as pandemic- and climate change-induced events. Limited research has examined the application of LHS concepts to emergency care settings. Implementation science should be utilised to better understand the factors influencing adoption of LHS approaches on the front lines of care, so that all five LHS components can be progressed in these settings.

The experiences of healthcare providers who refer to a campus-based pharmacy clinic: a qualitative analysis

Introduction The University of Otago School of Pharmacy Clinic (the Clinic) is a campus-based non-dispensing clinic that offers consultation-based medicines optimisation services to patients. Aim This project aims to understand the experiences and opinions of healthcareproviders who have referred patients to the School of Pharmacy Clinic, specifically: their motivation for referring patients; how the Clinic impacts providers, patients and the wider health system; provider satisfaction; and opportunities for further collaboration. Methods Semi-structured interviews were used to collect data from 15 participants who represented five health professions. An inductive reflexive thematic analysis approach was used to analyse the dataset from which codes and themes were developed. Normalisation Process Theory (NPT) was used to structure the interview guide and as a framework to present themes. Results Seven themes were developed; 'Perceptions of Pharmacists' (Coherence), 'Motivators for Engagement' and 'Barriers to Engagement' (Cognitive Participation), 'Utility of Pharmacist Feedback' and 'Opportunities' (Collective Action) and 'Referrers' Experiences' and 'Patient-centred Care' (Reflexive Action). Discussion Healthcare providers described predominantly positive experiences. Medically complex cases and patients requiring medicines education were most likely to be referred for consultation. Engaging with the Clinic presented valuable opportunities for interprofessional collaborative practice and continuing professional education. Referrers would like more regular contact with Clinic pharmacists to encourage interprofessional collaborative relationships. Patients were thought to benefit from their pharmacist's clinical expertise, time, patient-centred approach and subsequent medication and health optimisation. Integration of Clinic pharmacists into specialist outpatient clinics at Dunedin Hospital may broaden the scope and improve efficiency of their services.

Perspectives of people experiencing homelessness with recent non-fatal street drug overdose on the Pharmacist and Homeless Outreach Engagement and Non-medical Independent prescribing Rx (PHOENIx) intervention

INTRODUCTION

In Scotland, a third of all deaths of people experiencing homelessness (PExH) are street-drug-related, and less than half of their multiple physical- and mental health conditions are treated. New, holistic interventions are required to address these health inequalities. PHOENIx (Pharmacist Homeless Outreach Engagement and Non-medical Independent prescribing Rx) is delivered on outreach by National Health Service (NHS) pharmacist independent prescribers in partnership with third sector homelessness charity workers. We describe participant's perspectives of PHOENIx.

METHODS

This study aims to understand experiences of the PHOENIx intervention by participants recruited into the active arm of a pilot randomised controlled trial (RCT). Semi-structured in-person interviews explored participants' evaluation of the intervention. In this study, the four components (coherence, cognitive participation, collective action, reflexive monitoring) of the Normalisation Process Theory (NPT) framework underpinned data collection and analyses.

RESULTS

We identified four themes that were interpreted within the NPT framework that describe participant evaluation of the PHOENIx intervention: differentiating the intervention from usual care (coherence), embedding connection and consistency in practice (cognitive participation), implementation of practical and emotional operational work (collective action), and lack of power and a commitment to long-term support (reflexive monitoring). Participants successfully engaged with the intervention. Facilitators for participant motivation included the relationship-based work created by the PHOENIx team. This included operational work to fulfil both the practical and emotional needs of participants. Barriers included concern regarding power imbalances within the sector, a lack of long-term support and the impact of the intervention concluding.

CONCLUSIONS

Findings identify and describe participants' evaluations of the PHOENIx intervention. NPT is a theoretical framework facilitating understanding of experiences, highlighting both facilitators and barriers to sustained engagement and investment. Our findings inform future developments regarding a subsequent definitive RCT of PHOENIx, despite challenges brought about by challenging micro and macro-economic and political landscapes.

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.

METHODS

Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled.

RESULTS

A 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained.

CONCLUSION

A questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.

Exploring the role of professional identity in the implementation of clinical decision support systems-a narrative review

BACKGROUND

Clinical decision support systems (CDSSs) have the potential to improve quality of care, patient safety, and efficiency because of their ability to perform medical tasks in a more data-driven, evidence-based, and semi-autonomous way. However, CDSSs may also affect the professional identity of health professionals. Some professionals might experience these systems as a threat to their professional identity, as CDSSs could partially substitute clinical competencies, autonomy, or control over the care process. Other professionals may experience an empowerment of the role in the medical system. The purpose of this study is to uncover the role of professional identity in CDSS implementation and to identify core human, technological, and organizational factors that may determine the effect of CDSSs on professional identity.

METHODS

We conducted a systematic literature review and included peer-reviewed empirical studies from two electronic databases (PubMed, Web of Science) that reported on key factors to CDSS implementation and were published between 2010 and 2023. Our explorative, inductive thematic analysis assessed the antecedents of professional identity-related mechanisms from the perspective of different health care professionals (i.e., physicians, residents, nurse practitioners, pharmacists).

RESULTS

One hundred thirty-one qualitative, quantitative, or mixed-method studies from over 60 journals were included in this review. The thematic analysis found three dimensions of professional identity-related mechanisms that influence CDSS implementation success: perceived threat or enhancement of professional control and autonomy, perceived threat or enhancement of professional skills and expertise, and perceived loss or gain of control over patient relationships. At the technological level, the most common issues were the system's ability to fit into existing clinical workflows and organizational structures, and its ability to meet user needs. At the organizational level, time pressure and tension, as well as internal communication and involvement of end users were most frequently reported. At the human level, individual attitudes and emotional responses, as well as familiarity with the system, most often influenced the CDSS implementation. Our results show that professional identity-related mechanisms are driven by these factors and influence CDSS implementation success. The perception of the change of professional identity is influenced by the user's professional status and expertise and is improved over the course of implementation.

CONCLUSION

This review highlights the need for health care managers to evaluate perceived professional identity threats to health care professionals across all implementation phases when introducing a CDSS and to consider their varying manifestations among different health care professionals. Moreover, it highlights the importance of innovation and change management approaches, such as involving health professionals in the design and implementation process to mitigate threat perceptions. We provide future areas of research for the evaluation of the professional identity construct within health care.

Audit and feedback interventions involving pharmacists to influence prescribing behaviour in general practice: a systematic review and meta-analysis

INTRODUCTION

Pharmacists, as experts in medicines, are increasingly employed in general practices and undertake a range of responsibilities. Audit and feedback (A&F) interventions are effective in achieving behaviour change, including prescribing. The extent of pharmacist involvement in A&F interventions to influence prescribing is unknown. This review aimed to assess the effectiveness of A&F interventions involving pharmacists on prescribing in general practice compared with no A&F/usual care and to describe features of A&F interventions and pharmacist characteristics.

METHODS

Electronic databases (MEDLINE, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, (Social) Science Citation Indexes, ISI Web of Science) were searched (2012, 2019, 2020). Cochrane systematic review methods were applied to trial identification, selection, and risk of bias. Results were summarized descriptively and heterogeneity was assessed. A random-effects meta-analysis was conducted where studies were sufficiently homogenous in design and outcome.

RESULTS

Eleven cluster-randomized studies from 9 countries were included. Risk of bias across most domains was low. Interventions focussed on older patients, specific clinical area(s), or specific medications. Meta-analysis of 6 studies showed improved prescribing outcomes (pooled risk ratio: 0.78, 95% confidence interval: 0.64-0.94). Interventions including both verbal and written feedback or computerized decision support for prescribers were more effective. Pharmacists who received study-specific training, provided ongoing support to prescribers or reviewed prescribing for individual patients, contributed to more effective interventions.

CONCLUSIONS

A&F interventions involving pharmacists can lead to small improvements in evidence-based prescribing in general practice settings. Future implementation of A&F within general practice should compare different ways of involving pharmacists to determine how to optimize effectiveness.PRISMA-compliant abstract included in Supplementary Material 1.

Implementation strategies and outcome measures for advancing learning health systems: a mixed methods systematic review

BACKGROUND

Learning health systems strive to continuously integrate data and evidence into practice to improve patient outcomes and ensure value-based healthcare. While the LHS concept is gaining traction, the operationalization of LHSs is underexplored.

OBJECTIVE

To identify and synthesize the existing evidence on the implementation and evaluation of advancing learning health systems across international health care settings.

METHODS

A mixed methods systematic review was conducted. Six databases (CINAHL, Embase, Medline, PAIS, Scopus and Nursing at Allied Health Database) were searched up to July 2022 for terms related to learning health systems, implementation, and evaluation measures. Any study design, health care setting and population were considered for inclusion. No limitations were placed on language or date of publication. Two reviewers independently screened the titles, abstracts, and full texts of identified articles. Data were extracted and synthesized using a convergent integrated approach. Studies were critically appraised using relevant JBI critical appraisal checklists.

RESULTS

Thirty-five studies were included in the review. Most studies were conducted in the United States (n = 21) and published between 2019 and 2022 (n = 24). Digital data capture was the most common LHS characteristic reported across studies, while patient engagement, aligned governance and a culture of rapid learning and improvement were reported least often. We identified 33 unique strategies for implementing LHSs including: change record systems, conduct local consensus discussions and audit & provide feedback. A triangulation of quantitative and qualitative data revealed three integrated findings related to the implementation of LHSs: (1) The digital infrastructure of LHSs optimizes health service delivery; (2) LHSs have a positive impact on patient care and health outcomes; and (3) LHSs can influence health care providers and the health system.

CONCLUSION

This paper provides a comprehensive overview of the implementation of LHSs in various healthcare settings. While this review identified key implementation strategies, potential outcome measures, and components of functioning LHSs, further research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluation.

The use and role of digital technology in learning health systems: A scoping review

OBJECTIVE

The review aimed to identify which digital technologies are proposed or used within learning health systems (LHS) and to analyze the extent to which they support learning processes in LHS.

MATERIALS AND METHODS

Multiple databases and grey literature were searched with terms related to LHS. Manual searches and backward searches of reference lists were also undertaken. The review considered publications from 2007 to 2022. Records focusing on LHS, referring to one or more digital technologies, and describing how at least one digital technology could be used in LHS were included.

RESULTS

2046 records were screened for inclusion and 154 records were included in the analysis. Twenty categories of digital technology were identified. The two most common ones across records were data recording and processing and electronic health records. Digital technology was primarily leveraged to support data access and aggregation and data analysis, two of the seven recognized learning processes within LHS learning cycles.

DISCUSSION

The results of the review show that a wide array of digital technologies is being leveraged to support learning cycles within LHS. Nevertheless, an over-reliance on a narrow set of technologies supporting knowledge discovery, a lack of direct evaluation of digital technologies and ambiguity in technology descriptions are hindering the realization of the LHS vision.

CONCLUSION

Future LHS research and initiatives should aim to integrate digital technology to support practice change and impact evaluation. The use of recognized evaluation methods for health information technology and more detailed descriptions of proposed technologies are also recommended.

Using sociotechnical theory to understand medication safety work in primary care and prescribers' use of clinical decision support: a qualitative study

OBJECTIVES

The concept of safety work draws attention to the intentional work of ensuring safety within care systems. Clinical decision support (CDS) has been designed to enhance medication safety in primary care by providing decision-making support to prescribers. Sociotechnical theory understands that healthcare settings are complex and dynamically connected systems of fluid networks, human agents, changing relationships and social processes. This study aimed to understand the relationship between safety work and the use of CDS.

DESIGN AND SETTING

This qualitative study took place across nine different general practices in England. Stakeholders included general practitioners (GPs) and general practice-based pharmacists and nurse prescribers. Semi-structured interviews were conducted to illicit how the system was used by the participants in the context of medication safety work. Data analysis conducted alongside data collection was thematic and drew on socio-technical theory.

PARTICIPANTS

Twenty-three interviews were conducted with 14 GPs, three nurse prescribers and three practice pharmacists between February 2018 and June 2020.

RESULTS

Safety work was contextually situated in a complex network of relationships. Three interconnected themes were interpreted from the data: (1) the use of CDS within organisational and social practices and workflows; (2) safety work and the use of CDS within the interplay between prescribers, patients and populations; and (3) the affordances embedded in CDS systems.

CONCLUSION

The use of sociotechnical theory here extends current thinking in patient safety particularly in the ways that safety work was co-constituted with the use of CDS alerts. This has implications for implementation and use to ensure that the contexts into which such CDS systems are implemented are taken into account. Understanding how alerts can adapt safety culture will help improve the efficacy of CDS systems, enhance prescribing safety and help to further understand how safety work is achieved in primary care.

Understanding factors that could influence patient acceptability of the use of the PINCER intervention in primary care: A qualitative exploration using the Theoretical Framework of Acceptability

Introduction

Medication errors are an important cause of morbidity and mortality. The pharmacist-led IT-based intervention to reduce clinically important medication errors (PINCER) intervention was shown to reduce medication errors when tested in a cluster randomised controlled trial and when implemented across one region of England. Now that it has been rolled out nationally, and to enhance findings from evaluations with staff and stakeholders, this paper is the first to report patients’ perceived acceptability on the use of PINCER in primary care and proposes suggestions on how delivery of PINCER related care could be delivered in a way that is acceptable and not unnecessarily burdensome.

Methods

A total of 46 participants living with long-term health conditions who had experience of medication reviews and/or monitoring were recruited through patient participant groups and social media. Semi-structured, qualitative interviews and focus groups were conducted face-to-face or via telephone. A thematic analysis was conducted and findings mapped to the constructs of the Theoretical Framework of Acceptability (TFA).

Results

Two themes were identified and interpreted within the most relevant TFA construct: Perceptions on the purpose and components of PINCER (Affective Attitude and Intervention Coherence) and Perceived patient implications (Burden and Self-efficacy). Overall perceptions on PINCER were positive with participants showing good understanding of the components. Access to medication reviews, which PINCER related care can involve, was reported to be limited and a lack of consistency in practitioners delivering reviews was considered challenging, as was lack of communication between primary care and other health-care providers. Patients thought it would be helpful if medication reviews and prescription renewal times were synchronised. Remote medication review consultations were more convenient for some but viewed as a barrier to communication by others. It was acknowledged that some patients may be more resistant to change and more willing to accept changes initiated by general practitioners.

Conclusions

Participants found the concept of PINCER acceptable; however, acceptability could be improved if awareness on the role of primary care pharmacists is raised and patient-pharmacist relationships enhanced. Being transparent with communication and delivering streamlined and consistent but flexible PINCER related care is recommended.

Understanding factors influencing uptake and sustainable use of the PINCER intervention at scale: A qualitative evaluation using Normalisation Process Theory

Introduction

Medication errors are an important cause of morbidity and mortality. The pharmacist-led IT-based intervention to reduce clinically important medication errors (PINCER) has demonstrated improvements in primary care medication safety, and whilst now the subject of national roll-out its optimal and sustainable use across health contexts has not been fully explored. As part of a qualitative evaluation we aimed to identify factors influencing successful adoption, embedding and sustainable use of PINCER across primary care settings in England, UK.

Methods

Semi-structured face-to-face or telephone interviews, including follow-up interviews and an online survey were conducted with professionals knowledgeable of PINCER. Interview recruitment targeted four early adopter regions; the survey was distributed nationally. Initial data analysis was inductive, followed by analysis using a coding framework. A deductive matrix approach was taken to map the framework to the Normalisation Process Theory (NPT). Themes were then identified.

Results

Fifty participants were interviewed, 18 participated in a follow-up interview. Eighty-one general practices and three Clinical Commissioning Groups completed the survey. Four themes were identified and interpreted within the relevant NPT construct: Awareness & Perceptions (Coherence), Receptivity to PINCER (Cognitive Participation), Engagement [Collective Action] and Reflections & Adaptations (Reflexive Monitoring). Variability was identified in how PINCER awareness was raised and how staff worked to operationalise the intervention. Facilitators for use included stakeholder investment, favourable evidence, inclusion in policy, incentives, fit with individual and organisational goals and positive experiences. Barriers included lack of understanding, capacity concerns, operational difficulties and the impact of COVID-19. System changes such as adding alerts on clinical systems were indicative of embedding and continued use.

Conclusions

The NPT helped understand motives behind engagement and the barriers and facilitators towards sustainable use. Optimising troubleshooting support and encouraging establishments to adopt an inclusive approach to intervention adoption and utilisation could help accelerate uptake and help establish ongoing sustainable use.

Using a learning health system framework to examine COVID-19 pandemic planning and response at a Canadian Health Centre

Background

The COVID-19 pandemic has presented a unique opportunity to explore how health systems adapt under rapid and constant change and develop a better understanding of health system transformation. Learning health systems (LHS) have been proposed as an ideal structure to inform a data-driven response to a public health emergency like COVID-19. The aim of this study was to use a LHS framework to identify assets and gaps in health system pandemic planning and response during the initial stages of the COVID-19 pandemic at a single Canadian Health Centre.

Methods

This paper reports the data triangulation stage of a concurrent triangulation mixed methods study which aims to map study findings onto the LHS framework. We used a triangulation matrix to map quantitative (textual and administrative sources) and qualitative (semi-structured interviews) data onto the seven characteristics of a LHS and identify assets and gaps related to health-system receptors and research-system supports.

Results

We identified several health system assets within the LHS characteristics, including appropriate decision supports and aligned governance. Gaps were identified in the LHS characteristics of engaged patients and timely production and use of research evidence.

Conclusion

The LHS provided a useful framework to examine COVID-19 pandemic response measures. We highlighted opportunities to strengthen the LHS infrastructure for rapid integration of evidence and patient experience data into future practice and policy changes.

Effectiveness of clinical dashboards as audit and feedback or clinical decision support tools on medication use and test ordering: a systematic review of randomized controlled trials

BACKGROUND

Clinical dashboards used as audit and feedback (A&F) or clinical decision support systems (CDSS) are increasingly adopted in healthcare. However, their effectiveness in changing the behavior of clinicians or patients is still unclear. This systematic review aims to investigate the effectiveness of clinical dashboards used as CDSS or A&F tools (as a standalone intervention or part of a multifaceted intervention) in primary care or hospital settings on medication prescription/adherence and test ordering.

METHODS

Seven major databases were searched for relevant studies, from inception to August 2021. Two authors independently extracted data, assessed the risk of bias using the Cochrane RoB II scale, and evaluated the certainty of evidence using GRADE. Data on trial characteristics and intervention effect sizes were extracted. A narrative synthesis was performed to summarize the findings of the included trials.

RESULTS

Eleven randomized trials were included. Eight trials evaluated clinical dashboards as standalone interventions and provided conflicting evidence on changes in antibiotic prescribing and no effects on statin prescribing compared to usual care. Dashboards increased medication adherence in patients with inflammatory arthritis but not in kidney transplant recipients. Three trials investigated dashboards as part of multicomponent interventions revealing decreased use of opioids for low back pain, increased proportion of patients receiving cardiovascular risk screening, and reduced antibiotic prescribing for upper respiratory tract infections.

CONCLUSION

There is limited evidence that dashboards integrated into electronic medical record systems and used as feedback or decision support tools may be associated with improvements in medication use and test ordering.

A Framework for Evaluating Dashboards in Healthcare

In the era of 'information overload', effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place heavy requirements on dashboards as information systems in usability and effective design. Understanding these issues is challenging given the absence of consistent and comprehensive approaches to dashboard evaluation. In this article we systematically review literature on dashboard implementation in healthcare, where dashboards have been employed widely, and where there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the more subtle questions. We demonstrate the use of the proposed framework by a design study guided by this framework. We conclude by comparing this framework with existing literature, outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.

Implementing prescribing safety indicators in prisons: A mixed methods study

AIMS

To examine the prevalence of potentially hazardous prescribing in the prison setting using prescribing safety indicators (PSIs) and explore their implementation and use in practice.

METHODS

PSIs were identified and reviewed by the project team following a literature review and a nominal group discussion. Pharmacists at 2 prison sites deployed the PSIs using search protocols within their electronic health record. Prevalence rates and 95% confidence intervals (CIs) were generated for each indicator. Semi-structured interviews with 20 prison healthcare staff across England and Wales were conducted to explore the feasibility of deploying and using PSIs in prison settings.

RESULTS

Thirteen PSIs were successfully deployed mostly comprising drug-drug interactions (n = 9). Five yielded elevated prevalence rates: use of anticholinergics if aged ≥65 years (Site B: 25.8% [95%CI: 10.4-41.2%]), lack of antipsychotic monitoring for >12 months (Site A: 39.1% [95%CI: 27.1-52.1%]; Site B: 28.6% [95%CI: 17.9-41.4%]), prolonged use of hypnotics (Site B: 46.3% [95%CI: 35.6-57.1%]), antiplatelets prescribed with nonsteroidal anti-inflammatory drugs without gastrointestinal protection (Site A: 12.5% [95%CI: 0.0-35.4%]; Site B: 16.7% [95%CI: 0.4-64.1%]), and selective serotonin/norepinephrine reuptake inhibitors prescribed with nonsteroidal anti-inflammatory drugs/antiplatelets without gastrointestinal protection (Site A: 39.6% [95%CI: 31.2-48.4%]; Site B: 33.3% [95%CI: 20.8-47.9%]). Prison healthcare staff supported the use of PSIs and identified key considerations to guide its successful implementation, including staff engagement and PSI 'champions'. To respond to PSI searches, stakeholders suggested contextualised patient support through intraprofessional collaboration.

CONCLUSION

We successfully implemented a suite of PSIs into 2 prisons, identifying those with higher prevalence values as intervention targets. When appropriately resourced and integrated into staff workflow, PSI searches may support prescribing safety in prisons.

OUP accepted manuscript

Objectives

(1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes.

Methods

We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement.

Results

Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6–10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes.

Conclusions

e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans.

Protocol Registration

PROSPERO CRD42016048695.

Understanding the implementation, impact and sustainable use of an electronic pharmacy referral service at hospital discharge: A qualitative evaluation from a sociotechnical perspective

Introduction

The transition of patients across care settings is associated with a high risk of errors and preventable medication-related harm. Ensuring effective communication of information between health professionals is considered important for improving patient safety. A National Health Service(NHS) organisation in the North West of England introduced an electronic transfer of care around medicines (TCAM) system which enabled hospital pharmacists to send information about patient’s medications to their nominated community pharmacy. We aimed to understand the adoption, and the implications for sustainable use in practice of the TCAM service

Methods

We evaluated the TCAM service in a Clinical Commissioning Group (CCG) and NHS Foundation Trust in Salford, United Kingdom (UK). Participants were opportunistically recruited to take part in qualitative interviews through stakeholder networks and during hospital admission, and included hospital pharmacists, hospital pharmacy technicians, community pharmacists, general practice-based pharmacists, patients and their carers. A thematic analysis, that was iterative and concurrent with data collection, was undertaken using a template approach. The interpretation of the data was informed by broad sociotechnical theory.

Results

Twenty-three interviews were conducted with health care professionals patients and carers. The ways in which the newly implemented TCAM intervention was adopted and used in practice and the perceptions of it from different stakeholders were conceptualised into four main thematic areas: The nature of the network and how it contributed to implementation, use and sustainability; The material properties of the system; How work practices for medicines safety were adapted and evolved; and The enhancement of medication safety activities. The TCAM intervention was perceived as effective in providing community pharmacists with timely, more accurate and enhanced information upon discharge. This allowed for pharmacists to enhance clinical services designed to ensure that accurate medication reconciliation was completed, and the correct medication was dispensed for the patient.

Conclusions

By providing pharmacy teams with accurate and enhanced information the TCAM intervention supported healthcare professionals to establish and/or strengthen interprofessional networks in order to provide clinical services designed to ensure that accurate medication reconciliation and dispensing activities were completed. However, the intervention was implemented into a complex and at times fragmented network, and we recommend opportunities be explored to fully integrate this network to involve patients/carers, general practice pharmacists and two-way communication between primary and secondary care to further enhance the reach and impact of the TCAM service.

Strategies supporting sustainable prescribing safety improvement interventions in English primary care: a qualitative study

Background

While the use of prescribing safety indicators (PSI) can reduce potentially hazardous prescribing, there is a need to identify actionable strategies for the successful implementation and sustainable delivery of PSI-based interventions in general practice.

Aim

To identify strategies for the successful implementation and sustainable use of PSI-based interventions in routine primary care.

Design & setting

Qualitative study in primary care settings across England.

Method

Anchoring on a complex pharmacist-led IT-based intervention (PINCER) and clinical decision support (CDS) for prescribing and medicines management, a qualitative study was conducted using sequential, multiple methods. The methods comprised documentary analysis, semi-structured interviews, and online workshops to identify challenges and possible solutions to the longer-term sustainability of PINCER and CDS. Thematic analysis was used for the documentary analysis and stakeholder workshops, while template analysis was used for the semi-structured interviews. Findings across the three methods were synthesised using the RE-AIM (reach, efficacy, adoption, implementation, and maintenance) framework.

Results

Forty-eight documents were analysed, and 27 interviews and two workshops involving 20 participants were undertaken. Five main issues were identified, which aligned with the adoption and maintenance dimensions of RE-AIM: fitting into current context (adoption); engaging hearts and minds (maintenance); building resilience (maintenance); achieving engagement with secondary care (maintenance); and emphasising complementarity (maintenance).

Conclusion

Extending ownership of prescribing safety beyond primary care-based pharmacists, and achieving greater alignment between general practice and hospital prescribing safety initiatives, is fundamental to achieve sustained impact of PSI-based interventions in primary care.

Normalisation process theory and the implementation of a new glaucoma clinical pathway in hospital eye services: Perspectives of doctors, nurses and optometrists

BACKGROUND

Normalisation process theory reports the importance of contextual integration in successfully embedding novel interventions, with recent propositions detailing the role that 'plasticity' of intervention components and 'elasticity' of an intended setting contribute. We report on the introduction of a clinical pathway assessing patient non-responsiveness to treatment for glaucoma and ocular hypertension. The aim of this study was to assess the feasibility of implementing the Cardiff Model of Glaucoma Care into hospital eye services, identifying any issues of acceptability for staff through the filter of normalisation process theory.

METHODS

A prospective observational study was undertaken in four hospital eye services. This incorporated detailed qualitative semi-structured interviews with staff (n = 8) to gather their perceptions on the intervention's usefulness and practicality. In addition, observational field notes of patient and staff consultations (n = 88) were collected, as well as broader organisational observations from within the research sites (n = 52). Data collection and analysis was informed by the normalisation process theory framework.

RESULTS

Staff reported the pathway led to beneficial knowledge on managing patient treatment, but the model was sometimes perceived as overly prescriptive. This perception varied significantly based on the composition of clinics in relation to staff experience, staff availability and pre-existing clinical structures. The most commonly recounted barrier came in contextually integrating into sites where wider administrative systems were inflexible to intervention components.

CONCLUSIONS

Flexibility will be the key determinant of whether the clinical pathway can progress to wider implementation. Addressing the complexity and variation associated with practice between clinics required a remodelling of the pathway to maintain its central benefits but enhance its plasticity. Our study therefore helps to confirm propositions developed in relation to normalisation process theory, contextual integration, intervention plasticity, and setting elasticity. This enables the transferability of findings to healthcare settings other than ophthalmology, where any novel intervention is implemented.

The Indiana Learning Health System Initiative: Early experience developing a collaborative, regional learning health system

INTRODUCTION

Learning health systems (LHSs) are usually created and maintained by single institutions or healthcare systems. The Indiana Learning Health System Initiative (ILHSI) is a new multi-institutional, collaborative regional LHS initiative led by the Regenstrief Institute (RI) and developed in partnership with five additional organizations: two Indiana-based health systems, two schools at Indiana University, and our state-wide health information exchange. We report our experiences and lessons learned during the initial 2-year phase of developing and implementing the ILHSI.

METHODS

The initial goals of the ILHSI were to instantiate the concept, establish partnerships, and perform LHS pilot projects to inform expansion. We established shared governance and technical capabilities, conducted a literature review-based and regional environmental scan, and convened key stakeholders to iteratively identify focus areas, and select and implement six initial joint projects.

RESULTS

The ILHSI successfully collaborated with its partner organizations to establish a foundational governance structure, set goals and strategies, and prioritize projects and training activities. We developed and deployed strategies to effectively use health system and regional HIE infrastructure and minimize information silos, a frequent challenge for multi-organizational LHSs. Successful projects were diverse and included deploying a Fast Healthcare Interoperability Standards (FHIR)-based tool across emergency departments state-wide, analyzing free-text elements of cross-hospital surveys, and developing models to provide clinical decision support based on clinical and social determinants of health. We also experienced organizational challenges, including changes in key leadership personnel and varying levels of engagement with health system partners, which impacted initial ILHSI efforts and structures. Reflecting on these early experiences, we identified lessons learned and next steps.

CONCLUSIONS

Multi-organizational LHSs can be challenging to develop but present the opportunity to leverage learning across multiple organizations and systems to benefit the general population. Attention to governance decisions, shared goal setting and monitoring, and careful selection of projects are important for early success.

Learning health systems in primary care: a systematic scoping review

BACKGROUND

Learning health systems have been gaining traction over the past decade. The purpose of this study was to understand the spread of learning health systems in primary care, including where they have been implemented, how they are operating, and potential challenges and solutions.

METHODS

We completed a scoping review by systematically searching OVID Medline®, Embase®, IEEE Xplore®, and reviewing specific journals from 2007 to 2020. We also completed a Google search to identify gray literature.

RESULTS

We reviewed 1924 articles through our database search and 51 articles from other sources, from which we identified 21 unique learning health systems based on 62 data sources. Only one of these learning health systems was implemented exclusively in a primary care setting, where all others were integrated health systems or networks that also included other care settings. Eighteen of the 21 were in the United States. Examples of how these learning health systems were being used included real-time clinical surveillance, quality improvement initiatives, pragmatic trials at the point of care, and decision support. Many challenges and potential solutions were identified regarding data, sustainability, promoting a learning culture, prioritization processes, involvement of community, and balancing quality improvement versus research.

CONCLUSIONS

We identified 21 learning health systems, which all appear at an early stage of development, and only one was primary care only. We summarized and provided examples of integrated health systems and data networks that can be considered early models in the growing global movement to advance learning health systems in primary care.

The implementation, use and sustainability of a clinical decision support system for medication optimisation in primary care: A qualitative evaluation

Background

The quality and safety of prescribing in general practice is important, Clinical decision support (CDS) systems can be used which present alerts to health professionals when prescribing in order to identify patients at risk of potentially hazardous prescribing. It is known that such computerised alerts may improve the safety of prescribing in hospitals but their implementation and sustainable use in general practice is less well understood. We aimed to understand the factors that influenced the successful implementation and sustained use in primary care of a CDS system.

Methods

Participants were purposively recruited from Clinical Commissioning Groups (CCGs) and general practices in the North West and East Midlands regions of England and from the CDS developers. We conducted face-to-face and telephone-based semi-structured qualitative interviews with staff stakeholders. A selection of participants was interviewed longitudinally to explore the further sustainability 1–2 years after implementation of the CDS system. The analysis, informed by Normalisation Process Theory (NPT), was thematic, iterative and conducted alongside data collection.

Results

Thirty-nine interviews were conducted either individually or in groups, with 33 stakeholders, including 11 follow-up interviews. Eight themes were interpreted in alignment with the four NPT constructs: Coherence (The purpose of the CDS: Enhancing medication safety and improving cost effectiveness; Relationship of users to the technology; Engagement and communication between different stakeholders); Cognitive Participation (Management of the profile of alerts); Collective Action (Prescribing in general practice, patient and population characteristics and engagement with patients; Knowledge);and Reflexive Monitoring (Sustaining the use of the CDS through maintenance and customisation; Learning and behaviour change. Participants saw that the CDS could have a role in enhancing medication safety and in the quality of care. Engagement through communication and support for local primary care providers and management leaders was considered important for successful implementation. Management of prescribing alert profiles for general practices was a dynamic process evolving over time. At regional management levels, work was required to adapt, and modify the system to optimise its use in practice and fulfil local priorities. Contextual factors, including patient and population characteristics, could impact upon the decision-making processes of prescribers influencing the response to alerts. The CDS could operate as a knowledge base allowing prescribers access to evidence-based information that they otherwise would not have.

Conclusions

This qualitative evaluation utilised NPT to understand the implementation, use and sustainability of a widely deployed CDS system offering prescribing alerts in general practice. The system was understood as having a role in medication safety in providing relevant patient specific information to prescribers in a timely manner. Engagement between stakeholders was considered important for the intervention in ensuring prescribers continued to utilise its functionality. Sustained implementation might be enhanced by careful profile management of the suite of alerts in the system. Our findings suggest that the use and sustainability of the CDS was related to prescribers’ perceptions of the relevance of alerts. Shared understanding of the purpose of the CDS between CCGS and general practices particularly in balancing cost saving and safety messages could be beneficial.

Use of Visual Dashboards to Enhance Pharmacy Teaching

Teaching large cohorts of pharmacy students with a team of multiple tutors in a feedback intensive course poses challenges in relation the amount of data generated, data integrity, interpretation of the data and importantly application of the insights gained from the data. The dispensing and counselling course in the third year BPharm at the University of Sydney has implemented the USyd Pharmacy Dashboard, developed to address these challenges following the Technological Pedagogical Content Knowledge Framework (TPACK) to integrate technology into teaching. The dashboard was designed to improve the student experience through more consistent feedback, gain insights to improve teaching delivery and provide efficiencies in maintaining data integrity. The tool has been developed using an action-based research approach whereby ideas are put into practice as the means to further develop the idea and improve practice. Refinement of the USyd Pharmacy Dashboard over three years has shown improvements in teaching delivery as teachers can respond to emerging trends. Student performance and satisfaction scores have increased, mainly due to improved consistency between tutors and improved delivery of feedback. Time involved with administrative tasks such as data maintenance is reduced. Opportunities for further refinements such as real time benchmarking and developing an open learner model have become apparent.

Mixed methods evaluation of a computerised audit and feedback dashboard to improve patient safety through targeting acute kidney injury (AKI) in primary care

BACKGROUND

Reducing the harms associated with acute kidney injury (AKI) requires addressing a wide range of patient safety issues, including polypharmacy and transitions of care, particularly for vulnerable patient groups. Computerised audit and feedback can transform the way healthcare organisations measure, analyse and learn from quality and safety data across different care settings, potentially improving patient safety.

OBJECTIVE

To implement and evaluate an audit and feedback dashboard targeting AKI to improve patient safety, focusing on factors affecting a range of user characteristics in primary care.

METHODS

We performed a mixed methods study in three stages. Semi-structured interviews were initially performed with both primary (n = 10) and secondary care (n = 5) staff to gather user requirements for six quality indicators extracted from national guidance on post-discharge AKI care. Modified indicators were implemented in the Performance Improvement plaN GeneratoR (PINGR) audit and feedback dashboard for six months, across 45 general practices in Salford. Primary care professionals were then interviewed again (n = 7) and completed usability questionnaires. This was triangulated with an interrupted time series analysis on indicator performance, alongside software usage statistics.

RESULTS

Improvements were observed for the indicators for medication review (+9.01 %; 95 % Confidence Interval (CI), +6.95 % to +11.06 %) and blood pressure measurement (+5.20 %; 95 % CI + 3.61 % to +6.78 %). Variable performance and engagement were observed for other indicators including AKI coding (+0.39 %; 95 % CI -1.88 % to +2.65 %), serum creatinine (-3.40 %; 95 % CI -7.66 % to +0.85 %), proteinuria (-1.08 %; 95 % CI -1.47 % to +0.32 %) and providing patient information (+0.16 %; 95 % CI -0.41 % to +0.73 %). A key facilitator to engagement was the development of 'champions of change', achieved through a raised awareness of high-risk patients, guidelines, inconsistencies in coding practice and evidence for quality and safety performance. Barriers related to the specificity and perceived achievability of indicators, and limitations in resources.

CONCLUSION

In a six-month, quasi-experimental evaluation of an electronic audit and feedback dashboard targeting AKI, we found improvements for two out of six quality indicators. While information technology can facilitate improvements in patient safety, further allocation of protected staff time and investment into shared learning are needed to realise those improvements in practice.

Preventable medication harm across health care settings: a systematic review and meta-analysis

BACKGROUND

Mitigating or reducing the risk of medication harm is a global policy priority. But evidence reflecting preventable medication harm in medical care and the factors that derive this harm remain unknown. Therefore, we aimed to quantify the prevalence, severity and type of preventable medication harm across medical care settings.

METHODS

We performed a systematic review and meta-analysis of observational studies to compare the prevalence of preventable medication harm. Searches were carried out in Medline, Cochrane library, CINAHL, Embase and PsycINFO from 2000 to 27 January 2020. Data extraction and critical appraisal was undertaken by two independent reviewers. Random-effects meta-analysis was employed followed by univariable and multivariable meta-regression. Heterogeneity was quantified using the I² statistic, and publication bias was evaluated.

PROSPERO

CRD42020164156.

RESULTS

Of the 7780 articles, 81 studies involving 285,687 patients were included. The pooled prevalence for preventable medication harm was 3% (95% confidence interval (CI) 2 to 4%, I² = 99%) and for overall medication harm was 9% (95% CI 7 to 11%, I² = 99.5%) of all patient incidence records. The highest rates of preventable medication harm were seen in elderly patient care settings (11%, 95% 7 to 15%, n = 7), intensive care (7%, 4 to 12%, n = 6), highly specialised or surgical care (6%, 3 to 11%, n = 13) and emergency medicine (5%, 2 to 12%, n = 12). The proportion of mild preventable medication harm was 39% (28 to 51%, n = 20, I² = 96.4%), moderate preventable harm 40% (31 to 49%, n = 22, I² = 93.6%) and clinically severe or life-threatening preventable harm 26% (15 to 37%, n = 28, I² = 97%). The source of the highest prevalence rates of preventable harm were at the prescribing (58%, 42 to 73%, n = 9, I² = 94%) and monitoring (47%, 21 to 73%, n = 8, I² = 99%) stages of medication use. Preventable harm was greatest in medicines affecting the 'central nervous system' and 'cardiovascular system'.

CONCLUSIONS

This is the largest meta-analysis to assess preventable medication harm. We conclude that around one in 30 patients are exposed to preventable medication harm in medical care, and more than a quarter of this harm is considered severe or life-threatening. Our results support the World Health Organisation's push for the detection and mitigation of medication-related harm as being a top priority, whilst highlighting other key potential targets for remedial intervention that should be a priority focus for future research.

Implementing a social network intervention: can the context for its workability be created? A quasi-ethnographic study

Background

Policy makers and researchers recognise the challenges of implementing evidence-based interventions into routine practice. The process of implementation is particularly complex in local community environments. In such settings, the dynamic nature of the wider contextual factors needs to be considered in addition to capturing interactions between the type of intervention and the site of implementation throughout the process. This study sought to examine how networks and network formation influence the implementation of a self-management support intervention in a community setting.

Methods

An ethnographically informed approach was taken. Data collection involved obtaining and analysing documents relevant to implementation (i.e. business plan and health reports), observations of meetings and engagement events over a 28-month period and 1:1 interviews with implementation-network members. Data analysis utilised the adaptive theory approach and drew upon the Consolidated Framework for Implementation Research. The paper presents the implementation events in chronological order to illustrate the evolution of the implementation process.

Results

The implementation-network was configured from the provider-network and commissioning-network. The configuration of the implementation-network was influenced by both the alignment between the political landscape and the intervention, and also the intervention having a robust evidence base. At the outset of implementation, the network achieved stability as members were agreed on roles and responsibilities. The stability of the implementation-network was threatened as progress slowed. However, with a period of reflection and evaluation, and with a flexible and resilient network, implementation was able to progress.

Conclusions

Resilience and creativity of all involved in the implementation in community settings is required to engage with a process which is complex, dynamic, and fraught with obstacles. An implementation-network is required to be resilient and flexible in order to adapt to the dynamic nature of community contexts. Of particular importance is understanding the demands of the various network elements, and there is a requirement to pause for "reflection and evaluation" in order to modify the implementation process as a result of learning.

Atrial fibrillation dashboard evaluation using the think aloud protocol

BACKGROUND

Atrial fibrillation (AF) is a common cardiac arrhythmia which is a major risk factor for stroke, transient ischaemic attacks and increased mortality. Primary care management of AF can significantly reduce these risks. We carried out an evaluation to asses the usability of an AF dashboard developed to improve data quality and the quality of care.

METHOD

We developed an online dashboard about the quality of AF management for general practices of the Oxford Royal College of General Practitioners Research and Surveillance Centre network. The dashboard displays (1) case ascertainment, (2) a calculation of stroke and haemorrhage risk to assess whether the benefits of anticogulants outweigh their risk, (3) prescriptions of different types of anticoagulant and (4) if prescribed anticoagulant is at the correct dose. We conducted the think aloud evaluation, involving 24 dashboard users to improve its usability.

RESULTS

Analysis of 24 transcripts received produced 120 individual feedback items (ie, verbalised tasks) that were mapped across five usability problem classes. We enhanced the dashboard based on evaluation feedback to encourage adoption by general practices participating in the sentinel network.

CONCLUSIONS

The think aloud evaluation provided useful insights into important usability issues that require further development. Our enhanced AF dashboard was acceptable to clinicians and its impact on data quality and care should be assessed in a formal study.

Evaluation of a pharmacist-led actionable audit and feedback intervention for improving medication safety in UK primary care: An interrupted time series analysis

BACKGROUND

We evaluated the impact of the pharmacist-led Safety Medication dASHboard (SMASH) intervention on medication safety in primary care.

METHODS AND FINDINGS

SMASH comprised (1) training of clinical pharmacists to deliver the intervention; (2) a web-based dashboard providing actionable, patient-level feedback; and (3) pharmacists reviewing individual at-risk patients, and initiating remedial actions or advising general practitioners on doing so. It was implemented in 43 general practices covering a population of 235,595 people in Salford (Greater Manchester), UK. All practices started receiving the intervention between 18 April 2016 and 26 September 2017. We used an interrupted time series analysis of rates (prevalence) of potentially hazardous prescribing and inadequate blood-test monitoring, comparing observed rates post-intervention to extrapolations from a 24-month pre-intervention trend. The number of people registered to participating practices and having 1 or more risk factors for being exposed to hazardous prescribing or inadequate blood-test monitoring at the start of the intervention was 47,413 (males: 23,073 [48.7%]; mean age: 60 years [standard deviation: 21]). At baseline, 95% of practices had rates of potentially hazardous prescribing (composite of 10 indicators) between 0.88% and 6.19%. The prevalence of potentially hazardous prescribing reduced by 27.9% (95% CI 20.3% to 36.8%, p < 0.001) at 24 weeks and by 40.7% (95% CI 29.1% to 54.2%, p < 0.001) at 12 months after introduction of SMASH. The rate of inadequate blood-test monitoring (composite of 2 indicators) reduced by 22.0% (95% CI 0.2% to 50.7%, p = 0.046) at 24 weeks; the change at 12 months (23.5%) was no longer significant (95% CI -4.5% to 61.6%, p = 0.127). After 12 months, 95% of practices had rates of potentially hazardous prescribing between 0.74% and 3.02%. Study limitations include the fact that practices were not randomised, and therefore unmeasured confounding may have influenced our findings.

CONCLUSIONS

The SMASH intervention was associated with reduced rates of potentially hazardous prescribing and inadequate blood-test monitoring in general practices. This reduction was sustained over 12 months after the start of the intervention for prescribing but not for monitoring of medication. There was a marked reduction in the variation in rates of hazardous prescribing between practices.

Implementing, embedding and sustaining simulation-based education: What helps, what hinders

OBJECTIVES

Although there is much evidence to support the use of simulation-based education (SBE) in undergraduate education of health care professionals, less attention has been paid to how SBE, viewed as a complex intervention, is implemented and becomes embedded and sustained. This paper aims to explore factors that inhibited or promoted SBE becoming normal practice in undergraduate health care professional programmes.

METHODS

Participants involved in the organisation, design and delivery of SBE in the north of England were recruited purposefully from higher education institutions (HEI) and National Health Service (NHS) Trusts through local networks for qualitative telephone interviews. Transcripts were analysed inductively using a hybrid approach involving simultaneous inductive open coding and deductive coding using normalisation process theory (NPT) as a theoretical lens.

FINDINGS

A total of 12 NHS staff from 11 trusts and seven individuals from four HEIs were interviewed. There was considerable variation in the approach taken to implementation across organisations, which resulted in varying degrees of embeddedness. Implementation was challenged or enabled by organisational leadership, professional buy-in and the development and maturity of the strategic approach. Variation in understanding of the scope and pedagogical aims of SBE led to inequity between professions and organisations in investment and participation, as well as design and delivery of SBE.

CONCLUSIONS

Given the complexity of SBE, best practice in implementation should be considered fundamental to the successful delivery of SBE. The findings provide an explanation of how contextual factors can support or hinder implementation to maximise potential benefits and learning outcomes; this understanding can be used to better inform development of SBE strategies and highlight potential factors needed to navigate contextual barriers so that learning outcomes can be maximised.

Implementing a digital patient feedback system: an analysis using normalisation process theory

BACKGROUND

Patient feedback in the English NHS is now widespread and digital methods are increasingly used. Adoption of digital methods depends on socio-technical and contextual factors, alongside human agency and lived experience. Moreover, the introduction of these methods may be perceived as disruptive of organisational and clinical routines. The focus of this paper is on the implementation of a particular digital feedback intervention that was co-designed with health professionals and patients (the DEPEND study).

METHODS

The digital feedback intervention was conceptualised as a complex intervention and thus the study focused on the contexts within which it operated, and how the different participants made sense of the intervention and engaged with it (or not). Four health care sites were studied: an acute setting, a mental health setting, and two general practices. Qualitative data was collected through interviews and focus groups with professionals, patients and carers. In total 51 staff, 24 patients and 8 carers were included. Forty-two observations of the use of the digital feedback system were carried out in the four settings. Data analysis was based on modified grounded theory and Normalisation Process Theory (NPT) formed the conceptual framework.

RESULTS

Digital feedback made sense to health care staff as it was seen as attractive, fast to complete and easier to analyse. Patients had a range of views depending on their familiarity with the digital world. Patients mentioned barriers such as kiosk not being visible, privacy, lack of digital know-how, technical hitches with the touchscreen. Collective action in maintaining participation again differed between sites because of workload pressure, perceptions of roles and responsibilities; and in the mental health site major organisational change was taking place. For mental health service users, their relationship with staff and their own health status determined their digital use.

CONCLUSION

The potential of digital feedback was recognised but implementation should take local contexts, different patient groups and organisational leadership into account. Patient involvement in change and adaptation of the intervention was important in enhancing the embedding of digital methods in routine feedback. NPT allowed for a in-depth understanding of actions and interactions of both staff and patients.

Understanding the utilisation of a novel interactive electronic medication safety dashboard in general practice: a mixed methods study

BACKGROUND

Improving medication safety is a major concern in primary care settings worldwide. The Salford Medication safety dASHboard (SMASH) intervention provided general practices in Salford (Greater Manchester, UK) with feedback on their safe prescribing and monitoring of medications through an online dashboard, and input from practice-based trained clinical pharmacists. In this study we explored how staff working in general practices used the SMASH dashboard to improve medication safety, through interactions with the dashboard to identify potential medication safety hazards and their workflow to resolve identified hazards.

METHODS

We used a mixed-methods study design involving quantitative data from dashboard user interaction logs from 43 general practices during the first year of receiving the SMASH intervention, and qualitative data from semi-structured interviews with 22 pharmacists and physicians from 18 practices in Salford.

RESULTS

Practices interacted with the dashboard a median of 12.0 (interquartile range, 5.0-15.2) times per month during the first quarter of use to identify and resolve potential medication safety hazards, typically starting with the most prevalent hazards or those they perceived to be most serious. Having observed a potential hazard, pharmacists and practice staff worked together to resolve that in a sequence of steps (1) verifying the dashboard information, (2) reviewing the patient's clinical records, and (3) deciding potential changes to the patient's medicines. Over time, dashboard use transitioned towards regular but less frequent (median of 5.5 [3.5-7.9] times per month) checks to identify and resolve new cases. The frequency of dashboard use was higher in practices with a larger number of at-risk patients. In 24 (56%) practices only pharmacists used the dashboard; in 12 (28%) use by other practice staff increased as pharmacist use declined after the initial intervention period; and in 7 (16%) there was mixed use by both pharmacists and practice staff over time.

CONCLUSIONS

An online medication safety dashboard enabled pharmacists to identify patients at risk of potentially hazardous prescribing. They subsequently worked with GPs to resolve risks on a case-by-case basis, but there were marked variations in processes between some practices. Workload diminished over time as it shifted towards resolving new cases of hazardous prescribing.

Modelling the interactive behaviour of users with a medication safety dashboard in a primary care setting

OBJECTIVE

To characterise the use of an electronic medication safety dashboard by exploring and contrasting interactions from primary users (i.e. pharmacists) who were leading the intervention and secondary users (i.e. non-pharmacist staff) who used the dashboard to engage in safe prescribing practices.

MATERIALS AND METHODS

We conducted a 10-month observational study in which 35 health professionals used an instrumented medication safety dashboard for audit and feedback purposes in clinical practice as part of a wider intervention study. We modelled user interaction by computing features representing exploration and dwell time through user interface events that were logged on a remote database. We applied supervised learning algorithms to classify primary against secondary users.

RESULTS

We observed values for accuracy above 0.8, indicating that 80% of the time we were able to distinguish a primary user from a secondary user. In particular, the Multilayer Perceptron (MLP) yielded the highest values of precision (0.88), recall (0.86) and F-measure (0.86). The behaviour of primary users was distinctive in that they spent less time between mouse clicks (lower dwell time) on the screens showing the overview of the practice and trends. Secondary users exhibited a higher dwell time and more visual search activity (higher exploration) on the screens displaying patients at risk and visualisations.

DISCUSSION AND CONCLUSION

We were able to distinguish the interactive behaviour of primary and secondary users of a medication safety dashboard in primary care using timestamped mouse events. Primary users were more competent on population health monitoring activities, while secondary users struggled on activities involving a detailed breakdown of the safety of patients. Informed by these findings, we propose workflows that group these activities and adaptive nudges to increase user engagement.

Identifying potential prescribing safety indicators related to mental health disorders and medications: A systematic review

BACKGROUND

Prescribing errors and medication related harm may be common in patients with mental illness. However, there has been limited research focusing on the development and application of prescribing safety indicators (PSIs) for this population.

OBJECTIVE

Identify potential PSIs related to mental health (MH) medications and conditions.

METHODS

Seven electronic databases were searched (from 1990 to February 2019), including the bibliographies of included studies and of relevant review articles. Studies that developed, validated or updated a set of explicit medication-specific indicators or criteria that measured prescribing safety or quality were included, irrespective of whether they contained MH indicators or not. Studies were screened to extract all MH related indicators before two MH clinical pharmacists screened them to select potential PSIs based on established criteria. All indicators were categorised into prescribing problems and medication categories.

RESULTS

79 unique studies were included, 70 of which contained at least one MH related indicator. No studies were identified that focused on development of PSIs for patients with mental illness. A total of 1386 MH indicators were identified (average 20 (SD = 25.1) per study); 245 of these were considered potential PSIs. Among PSIs the most common prescribing problem was 'Potentially inappropriate prescribing considering diagnoses or conditions' (n = 91, 37.1%) and the lowest was 'omission' (n = 5, 2.0%). 'Antidepressant' was the most common PSI medication category (n = 85, 34.7%).

CONCLUSION

This is the first systematic review to identify a comprehensive list of MH related potential PSIs. This list should undergo further validation and could be used as a foundation for the development of new suites of PSIs applicable to patients with mental illness.