The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health information systems

Differences in changes of data completeness after the implementation of an electronic medical record in three surgical departments of a German hospital-a longitudinal comparative document analysis

PURPOSE

The European health data space promises an efficient environment for research and policy-making. However, this data space is dependent on high data quality. The implementation of electronic medical record systems has a positive impact on data quality, but improvements are not consistent across empirical studies. This study aims to analyze differences in the changes of data quality and to discuss these against distinct stages of the electronic medical record's adoption process.

METHODS

Paper-based and electronic medical records from three surgical departments were compared, assessing changes in data quality after the implementation of an electronic medical record system. Data quality was operationalized as completeness of documentation. Ten information that must be documented in both record types (e.g. vital signs) were coded as 1 if they were documented, otherwise as 0. Chi-Square-Tests were used to compare percentage completeness of these ten information and t-tests to compare mean completeness per record type.

RESULTS

A total of N = 659 records were analyzed. Overall, the average completeness improved in the electronic medical record, with a change from 6.02 (SD = 1.88) to 7.2 (SD = 1.77). At the information level, eight information improved, one deteriorated and one remained unchanged. At the level of departments, changes in data quality show expected differences.

CONCLUSION

The study provides evidence that improvements in data quality could depend on the process how the electronic medical record is adopted in the affected department. Research is needed to further improve data quality through implementing new electronical medical record systems or updating existing ones.

A dual analysis of bougie and stylet development trend and impact of Chinese regulations on medical devices innovation

Medical device research and development are characterized by high costs, extended timelines, inherent risks, and the necessity for interdisciplinary knowledge and skills. It is significantly influenced by policies, making the understanding of medical device innovation both important and challenging. This paper takes a dual approach to analyze medical device innovation. We reviewed representative clinical product of bougie and stylet and summarized the common characteristics and trend of these product. Innovations in these products often involve adding depth markings, replacing material and design structure, enhancing visualization, deciding between reusable or disposable designs, and integrating multi-functional features. This underscores the delicate balance between technological advancements and medical costs for widespread clinical applicability. We explored the guiding role of policy in medical device innovation, emphasizing its impact through an analysis of medical device regulations and policies in China. By offering insights from the perspectives of medical device companies and regulators, this paper aims to elucidate the critical aspects of medical device innovation, assisting researchers in mitigating risks during product development.

The challenge of adopting a collaborative information system for independent healthcare workers in France: a comprehensive study

This study aimed to explore the perception of an underutilised collaborative information system through qualitative research, utilizing semi-structured, in-depth interviews with independent midwives and physician. PROSPERO, is a collaborative information system designed to bridge the communication gap between community-based healthcare workers and hospital-based care teams for parturients in Lyon, France. Through 27 semi-structured in-depth interviews with midwives, obstetricians, and general practitioners, we identified key themes related to the system's adoption: implementation challenges, utilisation barriers, interprofessional dynamics, and hidden variables affecting system use. Participants recognised the potential of PROSPERO to improve information sharing and care coordination but expressed concerns about the system's integration into existing workflows, time constraints, and the need for adequate training and technical support. Interprofessional dynamics revealed differing perspectives between hospital and independent practitioners, emphasising the importance of trust-building and professional recognition. Hidden variables, such as hierarchical influences and confidentiality concerns, further complicated the system's adoption. Despite the consensus on the benefits of a collaborative information system, its implementation was hindered by mistrust between healthcare workers (i.e. between independent practitioners and hospital staff). Our findings suggest that fostering trust and addressing the identified barriers are crucial steps towards successful system implementation. The study contributes to understanding the complex interplay of factors influencing the adoption of collaborative healthcare technologies and highlights the need for strategies that support effective interprofessional collaboration and communication.ClinicalTrials ID NCT02593292.

Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting

OBJECTIVE

This study aims to understand whether higher use of a patient portal can have an impact on mental health functioning and recovery.

METHOD

A mixed methods approach was used for this study. In 2019-2021, patients with mental health diagnoses at outpatient clinics in an academic centre were invited to complete World Health Organization Disability Assessment Scale 12 (WHODAS-12) and Mental Health Recovery Measure surveys at baseline, 3 months, and 6 months after signing up for the portal. At the 3-month time point, patients were invited to a semistructured interview with a member of the team to contextualize the findings obtained from the surveys. Analytics data was also collected from the platform to understand usage patterns on the portal.

RESULTS

Overall, 113 participants were included in the analysis. There was no significant change in mental health functioning and recovery scores over the 6-month period. However, suboptimal usage was observed as 46% of participants did not complete any tasks within the portal. Thirty-five participants had low use of the portal (1-9 interactions) and 18 participants had high usage (10+ interactions). There were also no differences in mental health functioning and recovery scores between low and high users of the portal. Qualitative interviews highlighted many opportunities where the portal can support overall functioning and mental health recovery.

CONCLUSIONS

Collectively, this study suggests that higher use of a portal had no impact, either positive or negative, on mental health outcomes. While it may offer convenience and improved patient satisfaction, adequate support is needed to fully enable these opportunities for patient care. As the type of interaction with the portal was not specifically addressed, future work should focus on looking at ways to support patient engagement and portal usage throughout their care journey.

Access to automated comparative feedback reports in primary care - a study of intensity of use and relationship with clinical performance among Swedish primary care practices

BACKGROUND

Digital applications that automatically extract information from electronic medical records and provide comparative visualizations of the data in the form of quality indicators to primary care practices may facilitate local quality improvement (QI). A necessary condition for such QI to work is that practices actively access the data. The purpose of this study was to explore the use of an application that visualizes quality indicators in Swedish primary care, developed by a profession-led QI initiative ("Primärvårdskvalitet"). We also describe the characteristics of practices that used the application more or less extensively, and the relationships between the intensity of use and changes in selected performance indicators.

METHODS

We studied longitudinal data on 122 primary care practices' visits to pages (page views) in the application over a period up to 5 years. We compared high and low users, classified by the average number of monthly page views, with respect to practice and patient characteristics as well as baseline measurements of a subset of the performance indicators. We estimated linear associations between visits to pages with diabetes-related indicators and the change in measurements of selected diabetes indicators over 1.5 years.

RESULTS

Less than half of all practices accessed the data in a given month, although most practices accessed the data during at least one third of the observed months. High and low users were similar in terms of most studied characteristics. We found statistically significant positive associations between use of the diabetes indicators and changes in measurements of three diabetes indicators.

CONCLUSIONS

Although most practices in this study indicated an interest in the automated feedback reports, the intensity of use can be described as varying and on average limited. The positive associations between the use and changes in performance suggest that policymakers should increase their support of practices' QI efforts. Such support may include providing a formalized structure for peer group discussions of data, facilitating both understanding of the data and possible action points to improve performance, while maintaining a profession-led use of applications.

Evaluating the translation of implementation science to clinical artificial intelligence: a bibliometric study of qualitative research

Introduction

Whilst a theoretical basis for implementation research is seen as advantageous, there is little clarity over if and how the application of theories, models or frameworks (TMF) impact implementation outcomes. Clinical artificial intelligence (AI) continues to receive multi-stakeholder interest and investment, yet a significant implementation gap remains. This bibliometric study aims to measure and characterize TMF application in qualitative clinical AI research to identify opportunities to improve research practice and its impact on clinical AI implementation.

Methods

Qualitative research of stakeholder perspectives on clinical AI published between January 2014 and October 2022 was systematically identified. Eligible studies were characterized by their publication type, clinical and geographical context, type of clinical AI studied, data collection method, participants and application of any TMF. Each TMF applied by eligible studies, its justification and mode of application was characterized.

Results

Of 202 eligible studies, 70 (34.7%) applied a TMF. There was an 8-fold increase in the number of publications between 2014 and 2022 but no significant increase in the proportion applying TMFs. Of the 50 TMFs applied, 40 (80%) were only applied once, with the Technology Acceptance Model applied most frequently (n = 9). Seven TMFs were novel contributions embedded within an eligible study. A minority of studies justified TMF application (n = 51,58.6%) and it was uncommon to discuss an alternative TMF or the limitations of the one selected (n = 11,12.6%). The most common way in which a TMF was applied in eligible studies was data analysis (n = 44,50.6%). Implementation guidelines or tools were explicitly referenced by 2 reports (1.0%).

Conclusion

TMFs have not been commonly applied in qualitative research of clinical AI. When TMFs have been applied there has been (i) little consensus on TMF selection (ii) limited description of selection rationale and (iii) lack of clarity over how TMFs inform research. We consider this to represent an opportunity to improve implementation science's translation to clinical AI research and clinical AI into practice by promoting the rigor and frequency of TMF application. We recommend that the finite resources of the implementation science community are diverted toward increasing accessibility and engagement with theory informed practices. The considered application of theories, models and frameworks (TMF) are thought to contribute to the impact of implementation science on the translation of innovations into real-world care. The frequency and nature of TMF use are yet to be described within digital health innovations, including the prominent field of clinical AI. A well-known implementation gap, coined as the "AI chasm" continues to limit the impact of clinical AI on real-world care. From this bibliometric study of the frequency and quality of TMF use within qualitative clinical AI research, we found that TMFs are usually not applied, their selection is highly varied between studies and there is not often a convincing rationale for their selection. Promoting the rigor and frequency of TMF use appears to present an opportunity to improve the translation of clinical AI into practice.

Improving medication safety in a paediatric hospital: a mixed-methods evaluation of a newly implemented computerised provider order entry system

OBJECTIVES

Computerised provider order entry (CPOE) systems have been implemented around the world as a solution to reduce ordering and transcription errors. However, previous literature documented many challenges to attain this goal, especially in paediatric settings. The objectives of this study were to (1) analyse the impact of a paediatric CPOE system on medication safety and (2) suggest potential error prevention strategies.

METHODS

A pre-post observational study was conducted at the pilot ward (n=60 beds) of a paediatric academic health centre through mixed methods. The implementation project and medication management workflows were described through active participation to the project management team, observation, discussions and analysis of related documents. Furthermore, using incident reports, the nature of each error and error rate was compared between the preperiod and postperiod.

RESULTS

The global error rate was lower, but non-statistically significant, in the post implementation phase, which was mostly driven by a significant reduction in errors during order acknowledgement, transmission and transcription. Few errors occurred at the prescription step, and most errors occurred during medication administration. Furthermore, some errors could have been prevented using a CPOE in the pre-implementation period, and the CPOE led to few technology-related errors.

DISCUSSION AND CONCLUSION

This study identified both intended and unintended effects of CPOE adoption through the entire medication management workflow. This study revealed the importance of simplifying the acknowledgement, transmission and transcribing steps through the implementation of a CPOE to reduce medication errors. Improving the usability of the electronic medication administration record could help further improve medication safety.

How a point-of-care dashboard facilitates co-production of health care and health for and with individuals with psychotic disorders: a mixed-methods case study

BACKGROUND

Individuals with psychotic disorders experience widespread treatment failures and risk early death. Sweden's largest department specializing in psychotic disorders sought to improve patients' health by developing a point-of-care dashboard to support joint planning and co-production of care. The dashboard was tested for 18 months and included more than 400 patients at two outpatient clinics.

METHODS

This study evaluates the dashboard by addressing two questions: 1) Can differences in health-related outcome measures be attributed to the use of the dashboard? 2) How did the case managers experience the accessibility, use, and usefulness of the dashboard for co-producing care with individuals with psychotic disorders? This mixed-method case study used both Patient-Reported Outcome Measures (PROM) and data from a focus group interview with case managers. Data collection and analysis were framed by the Clinical Adoption Meta Model (CAMM) phases: i) accessibility, ii) system use, iii) behavior, and iv) clinical outcomes. The PROM used was the 12-item World Health Organization Disability Assessment Schedule (WHODAS 2.0), which assesses functional impairment and disability. Patients at clinics using the dashboard were matched with patients at clinics not using the dashboard. PROM data were compared using non-parametric statistics due to skewness in distribution. The focus group included five case managers who had experience using the dashboard with patients.

RESULTS

Compared to patients from clinics that did not use the dashboard, patients from clinics that did use the dashboard improved significantly overall (p = 0.045) and in the domain self-care (p = 0.041). Focus group participants reported that the dashboard supported data feedback-informed care and a proactive stance related to changes in patients' health. The dashboard helped users identify critical changes and enabled joint planning and evaluation.

CONCLUSION

Dashboard use was related to better patient health (WHODAS scores) when compared with matched patients from clinics that did not use the dashboard. In addition, case managers had a positive experience using the dashboard. Dashboard use might have lowered the risk for missing critical changes in patients' health while increasing the ability to proactively address needs. Future studies should investigate how to enhance patient co-production through use of supportive technologies.

Is primary health care ready for artificial intelligence? What do primary health care stakeholders say?

Background

Effective deployment of AI tools in primary health care requires the engagement of practitioners in the development and testing of these tools, and a match between the resulting AI tools and clinical/system needs in primary health care. To set the stage for these developments, we must gain a more in-depth understanding of the views of practitioners and decision-makers about the use of AI in primary health care. The objective of this study was to identify key issues regarding the use of AI tools in primary health care by exploring the views of primary health care and digital health stakeholders.

Methods

This study utilized a descriptive qualitative approach, including thematic data analysis. Fourteen in-depth interviews were conducted with primary health care and digital health stakeholders in Ontario. NVivo software was utilized in the coding of the interviews.

Results

Five main interconnected themes emerged: (1) Mismatch Between Envisioned Uses and Current Reality—denoting the importance of potential applications of AI in primary health care practice, with a recognition of the current reality characterized by a lack of available tools; (2) Mechanics of AI Don’t Matter: Just Another Tool in the Toolbox– reflecting an interest in what value AI tools could bring to practice, rather than concern with the mechanics of the AI tools themselves; (3) AI in Practice: A Double-Edged Sword—the possible benefits of AI use in primary health care contrasted with fundamental concern about the possible threats posed by AI in terms of clinical skills and capacity, mistakes, and loss of control; (4) The Non-Starters: A Guarded Stance Regarding AI Adoption in Primary Health Care—broader concerns centred on the ethical, legal, and social implications of AI use in primary health care; and (5) Necessary Elements: Facilitators of AI in Primary Health Care—elements required to support the uptake of AI tools, including co-creation, availability and use of high quality data, and the need for evaluation.

Conclusion

The use of AI in primary health care may have a positive impact, but many factors need to be considered regarding its implementation. This study may help to inform the development and deployment of AI tools in primary health care.

End-users feedback and perceptions associated with the implementation of a clinical-rule based Check of Medication Appropriateness service

Background

To support appropriate prescribing hospital-wide, the ‘Check of Medication Appropriateness’ (CMA) service was implemented at the University Hospitals Leuven. The CMA concerns a clinical rule based and pharmacist-led medication review service. The aim of this study was to explore both physicians’ and pharmacists’ feedback on the optimised CMA service to further improve the service.

Methods

An anonymous e-questionnaire was sent to all physicians active in the University Hospitals Leuven (n = 1631) and to all clinical pharmacists performing the CMA service (n = 16). Feedback was collected using multiple choice questions. During a 5-month period, physicians were also contacted in case of non-acceptance of recommendations to investigate barriers affecting implementation. Thematic analysis was performed and additional acceptance after telephone contact within 24 h was registered.

Results

A total of 119 physicians (7.3%) and 16 pharmacists (100%) completed the e-questionnaire. The overall service was assessed as clinically relevant to highly relevant by 77.7% of physicians. The main reasons for non-acceptance of recommendations were related to workload, work environment and time constraints. About two thirds (66.3%) of initially not-accepted recommendations were accepted after phone contact. A nearly full consensus was reached among pharmacists (15/16) on the centralised CMA being complementary to current clinical pharmacy activities. Two major barriers were reported by pharmacists: (1) too limited time allocation and (2) a large number of irrelevant alerts.

Conclusions

The CMA was perceived as clinically relevant by the majority of end-users. Acceptance rate of pharmaceutical recommendations was further increased by calling the physician. Increasing the specificity of clinical rules in the future is imperative.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01921-7.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

OBJECTIVE

Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation.

METHODS

The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies.

RESULTS

The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment.

DISCUSSION

Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation.

CONCLUSION

As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Team-based care Evaluation and Adoption Model (TEAM) Framework: Supporting the comprehensive evaluation of primary care transformation over time

OBJECTIVE

To introduce the new Team-based care Evaluation and Adoption Model (TEAM) Framework.

QUALITY OF EVIDENCE

The initial TEAM Framework was derived from a series of reviews and consultations with academic and clinical experts. In a parallel process, team-based primary and community care evaluation in Canada was assessed through a structured review of academic literature, followed by a review of policy literature of existing primary care evaluation frameworks.

MAIN MESSAGE

The review of academic articles alongside an analysis of policy documents and existing evaluation frameworks in primary care resulted in the development of the 10-dimension TEAM Framework.

CONCLUSION

Primary care transformation requires evaluation over time. The TEAM Framework provides a comprehensive framework for assessing evidence needed to support short- and long-term actionable improvements for team-based primary and community care in Canada. This framework will inform the development of an evaluation tool kit for primary care teams.

Approaches to Assess E-Health Programs: A Scoping Review

Context:

With the increasing trend of adopting e-Health technologies, the need for evidence for assessing e-Health technologies has become crucial. The appraisal of the e-Health program is important as this could provide guidance on further e-Health investment and adoption.

Aims:

The aim of the study was to provide an articulated body of literature on the current state of knowledge about the assessment of e-Health interventions.

Settings and Design:

Scoping review was conducted based on the framework provided by Arksey and O'Malley (2005) and considering enhancements proposed by Peters et al. (2015).

Subjects and Methods:

We searched the electronic databases and available gray literature from inception until the last week of October 2019. PRISMA flowchart for the study selection process was used to guide reporting. Data extraction included information on study design, authors, year of study, country, and key findings in terms of approaches used for assessment of e-Health programs. Data were compiled and summarized narratively.

Results:

Searches were performed between October and December 2019. Seventy-one relevant papers published between the years until the last week of October 2019 were reviewed and analyzed. After considering all the eligibility criteria, 15 papers were included. We identified 15 approaches for assessing e-Health programs, which were summarized and tabulated.

Conclusions:

This review showed that available literature on the assessment of e-Health programs is heterogeneous in terms of the methodology used. e-Health interventions are highly contextual; therefore; the phase of maturity and objective of the assessment should be considered while carrying out the assessment of e-Health programs.

Personal Electronic Records of Medications (PERMs) for medication reconciliation at care transitions: a rapid realist review

Background

Medication reconciliation (MedRec), a process to reduce medication error at care transitions, is labour- and resource-intensive and time-consuming. Use of Personal Electronic Records of Medications (PERMs) in health information systems to support MedRec have proven challenging. Relatively little is known about the design, use or implementation of PERMs at care transitions that impacts on MedRec in the ‘real world’. To respond to this gap in knowledge we undertook a rapid realist review (RRR). The aim was to develop theories to explain how, why, when, where and for whom PERMs are designed, implemented or used in practice at care transitions that impacts on MedRec.

Methodology

We used realist methodology and undertook the RRR between August 2020 and February 2021. We collaborated with experts in the field to identify key themes. Articles were sourced from four databases (Pubmed, Embase, CINAHL Complete and OpenGrey) to contribute to the theory development. Quality assessment, screening and data extraction using NVivo was completed. Contexts, mechanisms and outcomes configurations were identified and synthesised. The experts considered these theories for relevance and practicality and suggested refinements.

Results

Ten provisional theories were identified from 19 articles. Some theories relate to the design (T2 Inclusive design, T3 PERMs complement existing good processes, T7 Interoperability), some relate to the implementation (T5 Tailored training, T9 Positive impact of legislation or governance), some relate to use (T6 Support and on-demand training) and others relate iteratively to all stages of the process (T1 Engage stakeholders, T4 Build trust, T8 Resource investment, T10 Patients as users of PERMs).

Conclusions

This RRR has allowed additional valuable data to be extracted from existing primary research, with minimal resources, that may impact positively on future developments in this area. The theories are interdependent to a greater or lesser extent; several or all of the theories may need to be in play to collectively impact on the design, implementation or use of PERMs for MedRec at care transitions. These theories should now be incorporated into an intervention and evaluated to further test their validity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01659-8.

Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol

Background

The need for and usage of electronic patient records within hospitals has steadily increased over the last decade for economic reasons as well as the proceeding digitalization. While there are numerous benefits from this system, the potential risks of using electronic patient records for hospitals, patients and healthcare professionals must also be discussed. There is a lack in research, particularly regarding effects on healthcare professionals and their daily work in health services. The study eCoCo aims to gain insight into changes in interprofessional collaboration and clinical workflows resulting from introducing electronic patient records.

Methods

eCoCo is a multi-center case study integrating mixed methods from qualitative and quantitative social research. The case studies include three hospitals that undergo the process of introducing electronic patient records. Data are collected before and after the introduction of electronic patient records using participant observation, interviews, focus groups, time measurement, patient and employee questionnaires and a questionnaire to measure the level of digitalization. Furthermore, documents (patient records) as well as structural and administrative data are gathered. To analyze the interprofessional collaboration qualitative network analyses, reconstructive-hermeneutic analyses and document analyses are conducted. The workflow analyses, patient and employee assessment analyses and classification within the clinical adoption meta-model are conducted to provide insights into clinical workflows.

Discussion

This study will be the first to investigate the effects of introducing electronic patient records on interprofessional collaboration and clinical workflows from the perspective of healthcare professionals. Thereby, it will consider patients’ safety, legal and ethical concerns and quality of care. The results will help to understand the organization and thereby improve the performance of health services working with electronic patient records.

Trial registration

The study was registered at the German clinical trials register (DRKS00023343, Pre-Results) on November 17, 2020.

Factors affecting the mature use of electronic medical records by primary care physicians: a systematic review

BACKGROUND

Despite a substantial increase in the adoption of electronic medical records (EMRs) in primary health care settings, the use of advanced EMR features is limited. Several studies have identified both barriers and facilitating factors that influence primary care physicians' (PCPs) use of advanced EMR features and the maturation of their EMR use. The purpose of this study is to explore and identify the factors that impact PCPs' mature use of EMRs.

METHODS

A systematic review was conducted in accordance with the Cochrane Handbook. The MEDLINE, Embase, and PsycINFO electronic databases were searched from 1946 to June 13, 2019. Two independent reviewers screened the studies for eligibility; to be included, studies had to address factors influencing PCPs' mature use of EMRs. A narrative synthesis was conducted to collate study findings and to report on patterns identified across studies. The quality of the studies was also appraised.

RESULTS

Of the 1893 studies identified, 14 were included in this study. Reported factors that influenced PCPs' mature use of EMRs fell into one of the following 5 categories: technology, people, organization, resources, and policy. Concerns about the EMR system's functionality, lack of physician awareness of EMR functionality, limited physician availability to learn more about EMRs, the habitual use of successfully completing clinical tasks using only basic EMR features, business-oriented organizational objectives, lack of vendor training, limited resource availability, and lack of physician readiness were reported as barriers to PCPs' mature use of EMRs. The motivation of physicians, user satisfaction, coaching and peer mentoring, EMR experience, gender, physician perception, transition planning for changes in roles and work processes, team-based care, adequate technical support and training, sharing resources, practices affiliated with an integrated delivery system, financial incentives, and policies to increase EMR use all had a favorable impact on PCPs' use of advanced EMR features.

CONCLUSIONS

By using a narrative synthesis to synthesize the evidence, we identified interrelated factors influencing the mature use of EMRs by PCPs. The findings underline the need to provide adequate training and policies that facilitate the mature use of EMRs by PCPs.

TRIAL REGISTRATION

PROSPERO CRD42019137526.

International Comparison of Six Basic eHealth Indicators Across 14 Countries: An eHealth Benchmarking Study

BACKGROUND

Many countries adopt eHealth applications to support patient-centered care. Through information exchange, these eHealth applications may overcome institutional data silos and support holistic and ubiquitous (regional or national) information logistics. Available eHealth indicators mostly describe usage and acceptance of eHealth in a country. The eHealth indicators focusing on the cross-institutional availability of patient-related information for health care professionals, patients, and care givers are rare.

OBJECTIVES

This study aims to present eHealth indicators on cross-institutional availability of relevant patient data for health care professionals, as well as for patients and their caregivers across 14 countries (Argentina, Australia, Austria, Finland, Germany, Hong Kong as a special administrative region of China, Israel, Japan, Jordan, Kenya, South Korea, Sweden, Turkey, and the United States) to compare our indicators and the resulting data for the examined countries with other eHealth benchmarks and to extend and explore changes to a comparable survey in 2017. We defined "availability of patient data" as the ability to access data in and to add data to the patient record in the respective country.

METHODS

The invited experts from each of the 14 countries provided the indicator data for their country to reflect the situation on August 1, 2019, as date of reference. Overall, 60 items were aggregated to six eHealth indicators.

RESULTS

Availability of patient-related information varies strongly by country. Health care professionals can access patients' most relevant cross-institutional health record data fully in only four countries. Patients and their caregivers can access their health record data fully in only two countries. Patients are able to fully add relevant data only in one country. Finland showed the best outcome of all eHealth indicators, followed by South Korea, Japan, and Sweden.

CONCLUSION

Advancement in eHealth depends on contextual factors such as health care organization, national health politics, privacy laws, and health care financing. Improvements in eHealth indicators are thus often slow. However, our survey shows that some countries were able to improve on at least some indicators between 2017 and 2019. We anticipate further improvements in the future.

The State of Evidence in Patient Portals: Umbrella Review

BACKGROUND

Patient portals have emerged as a recognized digital health strategy. To date, research on patient portals has grown rapidly. However, there has been limited evaluation of the growing body of evidence on portal availability, use, clinical or health behavior and outcomes, and portal adoption over time.

OBJECTIVE

This paper aims to comprehensively consolidate the current state of evidence on patient portals using the umbrella review methodology, introduce our approach for evaluating evidence for quantitative and qualitative findings presented in included systematic reviews, and present a knowledge translation tool that can be used to inform all stages of patient portal adoption.

METHODS

For this study, a modified version of the Joanna Briggs Institute umbrella review method was used. Multiple databases were searched for systematic reviews focused on patient portals, and the final sample included 14 reviews. We conducted a meta-level synthesis of findings from quantitative, qualitative, and mixed methods primary studies reported in systematic reviews. We organized the umbrella review findings according to the Clinical Adoption Meta-Model (CAMM). Vote-counting, GRADE (Grading of Recommendations, Assessment, Development, and Evaluations), and CERQual (Confidence in the Evidence from Review of Qualitative Research) were used to assess the umbrella review evidence.

RESULTS

Our knowledge translation tool summarizes the findings in the form of an evidence map. Organized by the CAMM categories, the map describes the following factors that influence portal adoption and effects over time: patient contexts, patient's interest and satisfaction, portal design, facilitators and barriers, providers' attitudes, service utilization, behavioral effects, clinical outcomes, and patient-reported outcomes. The map lists the theories and mechanisms recognized in the included portal research while identifying the need for business models and organizational theories that can inform all stages of portal adoption. Our GRADE and CERQual umbrella review evaluation resulted in the majority of evidence being rated as moderate to low, which reflects methodological issues in portal research, insufficient number of studies, or mixed results in specific focus areas. The 2 findings with a high rating of evidence were patients' interest in using portals for communication and the importance of a simple display of information in the portals. Over 40 portal features were identified in the umbrella review, with communication through secure messaging and appointment booking mentioned in all systematic reviews.

CONCLUSIONS

Our umbrella review provides a meta-level synthesis to make sense of the evidence on patient portals from published systematic reviews. Unsystematic and variable reporting of portal features undermines the ability to evaluate and compare portal effects and overlooks the specific context of portal use. Research designs sensitive to the social, organizational, policy, and temporal dimensions are needed to better understand the underlying mechanisms and context that leverage the identified factors to improve portal use and effects.

Applying and Extending the FITT Framework to Identify the Challenges and Opportunities of Successful eHealth Services for Patient Self-Management: Qualitative Interview Study

Background

The number of public eHealth services that support patient self-management is rapidly increasing. However, the implementation of these eHealth services for self-management has encountered challenges.

Objective

The purpose of this paper was to analyze the challenges and opportunities of implementing eHealth services for self-management by focusing on the fit between the technical solution and clinical use.

Methods

We performed in-depth interviews with 10 clinical project coordinators and managers who were responsible for developing and implementing various eHealth services for self-management interventions in five university hospitals in Finland. The results were analyzed using content analysis and open coding. The Fit between Individuals, Task, and Technology (FITT) framework was used to interpret the findings.

Results

The implementation of self-management services involved many challenges related to technical problems, health professional acceptance, patient motivation, and health organization and management. The implementers identified practices to manage the identified challenges, including improving the design of the technology, supporting health professionals in the adoption of the eHealth services, changing the work processes and tasks, involving patients, and collectively planning the implementation inside an organization. The findings could be mostly attributed to the dimensions of the FITT framework.

Conclusions

The FITT framework helped to analyze the challenges related to the implementation, and most of them were related to poor fit. The importance of patients as stakeholders in eHealth services for patient self-management needs to be highlighted. Thus, we propose that patients should be added as a different type of individual dimension to the FITT framework. In addition, the framework could be extended to include organization and management in a new context dimension.

Surveying the Role of Analytics in Evaluating Digital Mental Health Interventions for Transition-Aged Youth: Scoping Review

BACKGROUND

Consumer-facing digital health interventions provide a promising avenue to bridge gaps in mental health care delivery. To evaluate these interventions, understanding how the target population uses a solution is critical to the overall validity and reliability of the evaluation. As a result, usage data (analytics) can provide a proxy for evaluating the engagement of a solution. However, there is paucity of guidance on how usage data or analytics should be used to assess and evaluate digital mental health interventions.

OBJECTIVE

This review aimed to examine how usage data are collected and analyzed in evaluations of mental health mobile apps for transition-aged youth (15-29 years).

METHODS

A scoping review was conducted using the Arksey and O'Malley framework. A systematic search was conducted on 5 journal databases using keywords related to usage and engagement, mental health apps, and evaluation. A total of 1784 papers from 2008 to 2019 were identified and screened to ensure that they included analytics and evaluated a mental health app for transition-aged youth. After full-text screening, 49 papers were included in the analysis.

RESULTS

Of the 49 papers included in the analysis, 40 unique digital mental health innovations were evaluated, and about 80% (39/49) of the papers were published over the past 6 years. About 80% involved a randomized controlled trial and evaluated apps with information delivery features. There were heterogeneous findings in the concept that analytics was ascribed to, with the top 3 being engagement, adherence, and acceptability. There was also a significant spread in the number of metrics collected by each study, with 35% (17/49) of the papers collecting only 1 metric and 29% (14/49) collecting 4 or more analytic metrics. The number of modules completed, the session duration, and the number of log ins were the most common usage metrics collected.

CONCLUSIONS

This review of current literature identified significant variability and heterogeneity in using analytics to evaluate digital mental health interventions for transition-aged youth. The large proportion of publications from the last 6 years suggests that user analytics is increasingly being integrated into the evaluation of these apps. Numerous gaps related to selecting appropriate and relevant metrics and defining successful or high levels of engagement have been identified for future exploration. Although long-term use or adoption is an important precursor to realizing the expected benefits of an app, few studies have examined this issue. Researchers would benefit from clarification and guidance on how to measure and analyze app usage in terms of evaluating digital mental health interventions for transition-aged youth. Given the established role of adoption in the success of health information technologies, understanding how to abstract and analyze user adoption for consumer digital mental health apps is also an emerging priority.

Effects of a Web-Based Patient Portal on Patient Satisfaction and Missed Appointment Rates: Survey Study

Background

Although electronic medical record (EMR)-tethered patient portals are common in other countries, they are still emerging in Canada.

Objective

We aimed to report user satisfaction and the effects of a patient portal on medical appointment attendance in a Canadian cohort of patients within our publicly funded health care system.

Methods

Two surveys were deployed, via email, at 2 weeks and 6 months following the first recorded patient portal access. Database audits of visit attendance were used to supplement and cross reference survey data.

Results

Between January 2016 and July 2018, 4296 patients accessed the patient portal. During the study, 28% (957/3421) consented patient portal users responded to one or more semistructured electronic surveys. Of respondents, 93% (891/957) reported that the patient portal was easy to use, 51% (492/975) reported it saved time when scheduling an appointment, and 40% (382/957) reported that they had to repeat themselves less during appointments. Respondents reported patient portal–related changes in health system use, with 48% (462/957) reporting avoiding a clinic visit and 2.7% (26/957) avoiding an emergency department visit. Across 19,968 visits in clinics where the patient portal was introduced, missed appointments were recorded in 9.5% (858/9021) of non–patient portal user visits, compared with 4.5% (493/9021) for patient portal users, representing a 53% relative reduction in no-show rates.

Conclusions

Early experience with an EMR-tethered patient portal showed strong reports of positive patient experience, a self-reported decrease in health system use, and a measured decrease in missed appointment rates. Implications on the expanded use of patient portals requires more quantitative and qualitative study in Canada.

Synthesising evidence on patient portals: a protocol for an umbrella review

INTRODUCTION

Over the last two decades, patient portals have emerged as a noticeable eHealth strategy. To date, research on patient portals has been rapidly increasing. Our umbrella review aims to provide a meta-level synthesis to make sense of the evidence on patient portals from published systematic reviews (SRs).

METHODS

We will employ a modified version of the Joanna Briggs Institute umbrella review method. The search strategy encompasses multiple databases. The inclusion criterion is specific to SRs focused on patient portal. Patients or public were not involved in this work.

ANALYSIS

Two researchers will independently screen titles/abstracts and then full-text articles against the inclusion/exclusion criteria. Methodological quality of included reviews will be assessed and data will be extracted from the final selection of reviews. These reviews will be categorised into quantitative, qualitative and/or mixed-synthesis groups based on information about the design of primary studies provided in the reviews. Correspondingly, we will create quantitative, qualitative and/or mixed-synthesis Excel data-extraction tables. Within each table, data will be extracted with the reference to primary studies as reported in the reviews, and will be synthesised into themes and then a smaller number of findings/outcomes. Modified Grading of Recommendations Assessment, Development and Evaluation (GRADE) and Confidence in the Evidence from Reviews of Qualitative research (CERQual) tools will be applied to assess the strength of evidence at the level of each finding/outcome. The output of our umbrella review will consist of summary of findings tables and evidence profile tables. A narrative meta-level synthesis will be provided. We will use the clinical adoption meta-model as an organising framework.

ETHICS AND DISSEMINATION

As an outcome of this review, we will create a guidance and roadmap to be used in a future Delphi study to gather feedback from Canadian eHealth stakeholders. We will also present at conferences and publish the final report. The umbrella review does not require ethical approval.

PROSPERO REGISTRATION NUMBER

CRD42018096657.

Design and validity of a questionnaire to assess national eHealth architecture (NEHA): a study protocol

INTRODUCTION

eHealth is critically important to build strong health systems, and accelerate the achievement of sustainable development goals, particularly universal health coverage. To support and strengthen the health system, the eHealth architecture needs to be formulated and established prior to the implementation and development of any national eHealth applications and services. The aim of this study is to design and validate a standard questionnaire to assess the current status of national eHealth architecture (NEHA) components.

METHODS AND ANALYSIS

This study will use a mixed-methods design consisting of four phases: (1) item generation through review of evidences and experts' opinions, (2) face and content validity of the questionnaire, (3) determination of a range of possible scenarios for each item included in the questionnaire and (4) evaluation of reliability. This questionnaire is expected to generate critical and important information about the status of NEHA components that will be useful for monitoring, formulating, developing, implementing and evaluating NEHA. Our paper will contribute, we envisage, to establishment of a socio-technical basis on which governments and other relevant sectors can compare the policy interventions that boost the availability and utilisation of eHealth services within their settings.

ETHICS AND DISSEMINATION

The Ethics Committee for Research at the Tehran University of Medical Sciences approved the study protocol. We will obtain informed consent from each participant and collect data anonymously to maintain confidentiality. The translation of the findings into future policy planning will include the production of a series of peer-reviewed articles, presentation of the findings at relevant eHealth conferences and preparation of policy reports to the international organisations aiming to strengthen national capacity for better-informed eHealth architecture.

Primary care physicians' attitudes to the adoption of electronic medical records: a systematic review and evidence synthesis using the clinical adoption framework

BACKGROUND

Recent decades have seen rapid growth in the implementation of Electronic Medical Records (EMRs) in healthcare settings in both developed regions as well as low and middle income countries. Yet despite substantial investment, the implementation of EMRs in some primary care systems has lagged behind other settings, with piecemeal adoption of EMR functionality by primary care physicians (PCPs) themselves. We aimed to review and synthesise international literature on the attitudes of PCPs to EMR adoption using the Clinical Adoption (CA) Framework.

METHODS

MEDLINE, PsycINFO, and EMBASE were searched from 1st January 1996 to 1st August 2017 for studies investigating PCP attitudes towards EMR adoption. Papers were screened by two independent reviewers, and eligible studies selected for further assessment. Findings were categorised against the CA Framework and the quality of studies assessed against one of three appropriate tools.

RESULTS

Out of 2263 potential articles, 33 were included, based in North and South America, Europe, Middle East and Hong Kong. Concerns about the accessibility, reliability and EMR utility exerted an adverse influence on PCPs' attitudes to adoption. However many were positive about their potential to improve clinical productivity, patient safety and care quality. Younger, computer-literate PCPs, based in large/multi-group practices, were more likely to be positively inclined to EMR use than older physicians, less-skilled in technology use, based in solo practices. Adequate training, policies and procedures favourably impacted on PCPs' views on EMR implementation. Financial factors were common system level influencers shaping EMR adoption, from start-up costs to the resources required by ongoing use.

CONCLUSIONS

By using the CA Framework to synthesise the evidence, we identified a linked series of factors influencing PCPs attitudes to EMR adoption. Findings underline the need to involve end-users in future implementation programmes from the outset, to avoid the development of an EMR which is neither feasible nor acceptable for use in practice.

TRIAL REGISTRATION

PROSPERO CRD42016038790 .

Towards optimal electronic medical record use: perspectives of advanced users

BACKGROUND

While primary health care electronic medical record (EMR) adoption has increased in Canada, the use of advanced EMR features is limited. Realizing the potential benefits of primary health care EMR use is dependent not only on EMR acquisition, but also on its comprehensive use and integration into practice; yet, little is known about the advanced use of EMRs in primary health care.

OBJECTIVE

To explore the views of advanced primary health care EMR users practising in a team-based environment.

METHODS

A descriptive qualitative approach was used to explore the views of primary health care practitioners who were identified as advanced EMR users. Twelve individual semi-structured interviews were held with primary health care practitioners in Southwestern Ontario, Canada. Field notes were created after each interview. Interviews were audio recorded and transcribed verbatim. Researchers independently coded the transcripts and then met to discuss the results of the coding. We used a thematic approach to data analysis.

RESULTS

Three themes emerged from the data analysis: advanced EMR users as individuals with signature characteristics, advanced EMR users as visionaries and advanced EMR users as agents of change. In any one participant, these elements could overlap, illuminating the important interplay between these themes. Taken together, these themes defined advanced use among this group of primary health care practitioners.

CONCLUSIONS

To realize the potential benefits of EMR use in improved patient care and outcomes, we need to understand how to support EMR use. This study provides a necessary building block in furthering this understanding.

Applying STOPP Guidelines in Primary Care Through Electronic Medical Record Decision Support: Randomized Control Trial Highlighting the Importance of Data Quality

Background

Potentially Inappropriate Prescriptions (PIPs) are a common cause of morbidity, particularly in the elderly.

Objective

We sought to understand how the Screening Tool of Older People’s Prescriptions (STOPP) prescribing criteria, implemented in a routinely used primary care Electronic Medical Record (EMR), could impact PIP rates in community (non-academic) primary care practices.

Methods

We conducted a mixed-method, pragmatic, cluster, randomized control trial in research naïve primary care practices. Phase 1: In the randomized controlled trial, 40 fully automated STOPP rules were implemented as EMR alerts during a 16-week intervention period. The control group did not receive the 40 STOPP rules (but received other alerts). Participants were recruited through the OSCAR EMR user group mailing list and in person at user group meetings. Results were assessed by querying EMR data PIPs. EMR data quality probes were included. Phase 2: physicians were invited to participate in 1-hour semi-structured interviews to discuss the results.

Results

In the EMR, 40 STOPP rules were successfully implemented. Phase 1: A total of 28 physicians from 8 practices were recruited (16 in intervention and 12 in control groups). The calculated PIP rate was 2.6% (138/5308) (control) and 4.11% (768/18,668) (intervention) at baseline. No change in PIPs was observed through the intervention (P=.80). Data quality probes generally showed low use of problem list and medication list. Phase 2: A total of 5 physicians participated. All the participants felt that they were aware of the alerts but commented on workflow and presentation challenges.

Conclusions

The calculated PIP rate was markedly less than the expected rate found in literature (2.6% and 4.0% vs 20% in literature). Data quality probes highlighted issues related to completeness of data in areas of the EMR used for PIP reporting and by the decision support such as problem and medication lists. Users also highlighted areas for better integration of STOPP guidelines with prescribing workflows. Many of the STOPP criteria can be implemented in EMRs using simple logic. However, data quality in EMRs continues to be a challenge and was a limiting step in the effectiveness of the decision support in this study. This is important as decision makers continue to fund implementation and adoption of EMRs with the expectation of the use of advanced tools (such as decision support) without ongoing review of data quality and improvement.

Trial Registration

Clinicaltrials.gov NCT02130895; https://clinicaltrials.gov/ct2/show/NCT02130895 (Archived by WebCite at http://www.webcitation.org/6qyFigSYT)

Conceptual Models in Health Informatics Research: A Literature Review and Suggestions for Development

BACKGROUND

Contributing to health informatics research means using conceptual models that are integrative and explain the research in terms of the two broad domains of health science and information science. However, it can be hard for novice health informatics researchers to find exemplars and guidelines in working with integrative conceptual models.

OBJECTIVES

The aim of this paper is to support the use of integrative conceptual models in research on information and communication technologies in the health sector, and to encourage discussion of these conceptual models in scholarly forums.

METHODS

A two-part method was used to summarize and structure ideas about how to work effectively with conceptual models in health informatics research that included (1) a selective review and summary of the literature of conceptual models; and (2) the construction of a step-by-step approach to developing a conceptual model.

RESULTS

The seven-step methodology for developing conceptual models in health informatics research explained in this paper involves (1) acknowledging the limitations of health science and information science conceptual models; (2) giving a rationale for one's choice of integrative conceptual model; (3) explicating a conceptual model verbally and graphically; (4) seeking feedback about the conceptual model from stakeholders in both the health science and information science domains; (5) aligning a conceptual model with an appropriate research plan; (6) adapting a conceptual model in response to new knowledge over time; and (7) disseminating conceptual models in scholarly and scientific forums.

CONCLUSIONS

Making explicit the conceptual model that underpins a health informatics research project can contribute to increasing the number of well-formed and strongly grounded health informatics research projects. This explication has distinct benefits for researchers in training, research teams, and researchers and practitioners in information, health, and other disciplines.

Analysis of the factors influencing healthcare professionals' adoption of mobile electronic medical record (EMR) using the unified theory of acceptance and use of technology (UTAUT) in a tertiary hospital

Background

Although the factors that affect the end-user’s intention to use a new system and technology have been researched, the previous studies have been theoretical and do not verify the factors that affected the adoption of a new system. Thus, this study aimed to confirm the factors that influence users’ intentions to utilize a mobile electronic health records (EMR) system using both a questionnaire survey and a log file analysis that represented the real use of the system.

Methods

After observing the operation of a mobile EMR system in a tertiary university hospital for seven months, we performed an offline survey regarding the user acceptance of the system based on the Unified Theory of Acceptance and Use of Technology (UTAUT) and the Technology Acceptance Model (TAM). We surveyed 942 healthcare professionals over two weeks and performed a structural equation modeling (SEM) analysis to identify the intention to use the system among the participants. Next, we compared the results of the SEM analysis with the results of the analyses of the actual log files for two years to identify further insights into the factors that affected the intention of use. For these analyses, we used SAS 9.0 and AMOS 21.

Results

Of the 942 surveyed end-users, 48.3 % (23.2 % doctors and 68.3 % nurses) responded. After eliminating six subjects who completed the survey insincerely, we conducted the SEM analyses on the data from 449 subjects (65 doctors and 385 nurses). The newly suggested model satisfied the standards of model fitness, and the intention to use it was especially high due to the influences of Performance Expectancy on Attitude and Attitude. Based on the actual usage log analyses, both the doctors and nurses used the menus to view the inpatient lists, alerts, and patients’ clinical data with high frequency. Specifically, the doctors frequently retrieved laboratory results, and the nurses frequently retrieved nursing notes and used the menu to assume the responsibilities of nursing work.

Conclusion

In this study, the end-users’ intentions to use the mobile EMR system were particularly influenced by Performance Expectancy and Attitude. In reality, the usage log revealed high-frequency use of the functions to improve the continuity of care and work efficiency. These results indicate the influence of the factor of performance expectancy on the intention to use the mobile EMR system. Consequently, we suggest that when determining the implementation of mobile EMR systems, the functions that are related to workflow with ability to increase performance should be considered first.

Findings from the Clinical Information Systems Perspective

OBJECTIVE

To summarize recent research and to propose a selection of best papers published in 2014 in the field of Clinical Information Systems (CIS).

METHOD

A query with search terms from the Medical Subject Headings (MeSH) catalog as well as additional free text search terms was designed to identify relevant publications in the field of clinical information systems from PubMed and Web of Science®. The retrieved articles were then categorized in a multi-pass review carried out separately by the section editors. The final selection of 15 candidate papers was then peerreviewed by Yearbook editors and external reviewers. Based on the review results the four best papers were then selected at the best papers selection meeting with the IMIA Yearbook editorial board.

RESULTS

The query was carried out in mid-January 2015, yielding a combined result set of 1525 articles which were published in 722 different journals. Among these articles two main thematic sections were identified: i) Interoperability from a syntactical and semantic point of view as well as from a longterm preservation and organizational/legal point of view and ii) secondary use of existing health data in all its shades. Here, patient safety was a major scope of application.

CONCLUSIONS

CIS have become mature over the last years. The focus has now moved beyond data acquisition for just supporting the local care workflows. Actual research efforts in the CIS domain comprise the breakdown of information silos, the reduction of barriers between different systems of different care providers and secondary use of accumulated health data for multiple purposes.