A Survey of Clinicians' Views of the Utility of Large Language Models

OBJECTIVES

 Large language models (LLMs) like Generative pre-trained transformer (ChatGPT) are powerful algorithms that have been shown to produce human-like text from input data. Several potential clinical applications of this technology have been proposed and evaluated by biomedical informatics experts. However, few have surveyed health care providers for their opinions about whether the technology is fit for use.

METHODS

 We distributed a validated mixed-methods survey to gauge practicing clinicians' comfort with LLMs for a breadth of tasks in clinical practice, research, and education, which were selected from the literature.

RESULTS

 A total of 30 clinicians fully completed the survey. Of the 23 tasks, 16 were rated positively by more than 50% of the respondents. Based on our qualitative analysis, health care providers considered LLMs to have excellent synthesis skills and efficiency. However, our respondents had concerns that LLMs could generate false information and propagate training data bias.Our survey respondents were most comfortable with scenarios that allow LLMs to function in an assistive role, like a physician extender or trainee.

CONCLUSION

 In a mixed-methods survey of clinicians about LLM use, health care providers were encouraging of having LLMs in health care for many tasks, and especially in assistive roles. There is a need for continued human-centered development of both LLMs and artificial intelligence in general.

Visualising Variation in the Real-World Clinical Delivery of Chemotherapy Protocols

Typical univariate measures of variation in chemotherapy protocols fail to capture and describe the full multi-dimensional complexity of treatment adjustments in real-world data. In this preliminary work, we propose novel visualisations of observed treatment events, as well as treatment-as-delivered relative to initial prescriptions, as a means of gaining insights into complex patterns of treatment variation in cancer patients. Simple clustering techniques were also used to confirm the utility of these visualisations and our ability to correlate observed variations with historical events.

Interpreting Laboratory Results with Complementary Health Information: A Human Factors Perspective

The desire to access personal and high-quality health information electronically is increasing, not only in Canada, but globally. With the advent of the COVID - 19 pandemic the desire and demand for telemedicine and timely access to personal health data such as online laboratory (lab) results has increased substantially. This study examines citizens' perspectives of being provided with high-quality information about a specific lab test (i.e., potassium) in the same display as a trend graph. Therefore, the objective of this study is to test how participants managed this additional information about the context of the test, understood, and applied it. The researchers analyzed the responses of semi-structured interviews with Canadian participants (N=24) using conventional content analysis. This paper examined four themes related to providing complementary information concurrently with lab results in the same display: 1) Benefits of Collocated Information, 2) Information Overload, 3) Misinterpretation, 4) Confusion. This study provided examples of some of the difficulties that the participants faced accessing their lab values online, while navigating and discerning complimentary high-quality health information available in their patient portal.

Avenues for Strengthening PCORnet's Capacity to Advance Patient-Centered Economic Outcomes in Patient-Centered Outcomes Research (PCOR)

PCORnet, the National Patient-Centered Clinical Research Network, provides the ability to conduct prospective and observational pragmatic research by leveraging standardized, curated electronic health records data together with patient and stakeholder engagement. PCORnet is funded by the Patient-Centered Outcomes Research Institute (PCORI) and is composed of 8 Clinical Research Networks that incorporate at total of 79 health system "sites." As the network developed, linkage to commercial health plans, federal insurance claims, disease registries, and other data resources demonstrated the value in extending the networks infrastructure to provide a more complete representation of patient's health and lived experiences. Initially, PCORnet studies avoided direct economic comparative effectiveness as a topic. However, PCORI's authorizing law was amended in 2019 to allow studies to incorporate patient-centered economic outcomes in primary research aims. With PCORI's expanded scope and PCORnet's phase 3 beginning in January 2022, there are opportunities to strengthen the network's ability to support economic patient-centered outcomes research. This commentary will discuss approaches that have been incorporated to date by the network and point to opportunities for the network to incorporate economic variables for analysis, informed by patient and stakeholder perspectives. Topics addressed include: (1) data linkage infrastructure; (2) commercial health plan partnerships; (3) Medicare and Medicaid linkage; (4) health system billing-based benchmarking; (5) area-level measures; (6) individual-level measures; (7) pharmacy benefits and retail pharmacy data; and (8) the importance of transparency and engagement while addressing the biases inherent in linking real-world data sources.

Implementing Clinical Information Systems in Sub-Saharan Africa: Report and Lessons Learned From the MatLook Project in Cameroon

BACKGROUND

Yaoundé Central Hospital (YCH), located in the capital of Cameroon, is one of the leading referral hospitals in Cameroon. The hospital has several departments, including the Department of Gynecology-Obstetrics (hereinafter referred to as "the Maternity"). This clinical department has faced numerous problems with clinical information management, including the lack of high-quality and reliable clinical information, lack of access to this information, and poor use of this information.

OBJECTIVE

We aim to improve the management of clinical information generated at the Maternity at YCH and to describe the challenges, success factors, and lessons learned during its implementation and use.

METHODS

Based on an open-source hospital information system (HIS), this intervention implemented a clinical information system (CIS) at the Maternity at YCH and was carried out using the HERMES model-the first part aimed to cover outpatient consultations, billing, and cash management of the Maternity. Geneva University Hospitals supported this project, and several outcomes were measured at the end. The following outcomes were assessed: project management, technical and organizational aspects, leadership, change management, user training, and system use.

IMPLEMENTATION (RESULTS)

The first part of the project was completed, and the CIS was deployed in the Maternity at YCH. The main technical activities were adapting the open-source HIS to manage outpatient consultations and develop integrated billing and cash management software. In addition to technical aspects, we implemented several other activities. They consisted of the implementation of appropriate project governance or management, improvement of the organizational processes at the Maternity, promotion of the local digital health leadership and performance of change management, and implementation of the training and support of users. Despite barriers encountered during the project, the 6-month evaluation showed that the CIS was effectively used during the first 6 months.

CONCLUSIONS

Implementation of the HIS or CIS is feasible in a resource-limited setting such as Cameroon. The CIS was implemented based on good practices at the Maternity at YCH. This project had successes but also many challenges. Beyond project management and technical and financial aspects, the other main problems of implementing health information systems or HISs in Africa lie in digital health leadership, governance, and change management. This digital health leadership, governance, and change management should prioritize data as a tool for improving productivity and managing health institutions, and promote a data culture among health professionals to support a change in mindset and the acquisition of information management skills. Moreover, in countries with a highly centralized political system like ours, a high-level strategic and political anchor for such projects is often necessary to guarantee their success.

The practicalities of adapting UK maternity clinical information systems for observational research: Experiences of the POOL study

Background

Using routinely collected clinical data for observational research is an increasingly important method for data collection, especially when rare outcomes are being explored. The POOL study was commissioned to evaluate the safety of waterbirth in the UK using routine maternity and neonatal clinical data. This paper describes the design, rationale, set-up and pilot for this data linkage study using bespoke methods.

Methods

Clinical maternity information systems hold many data items of value for research purposes, but often lack specific data items required for individual studies. This study used the novel method of amending an existing clinical maternity database for the purpose of collecting additional research data fields. In combination with the extraction of existing data fields, this maximised the potential use of existing routinely collected clinical data for research purposes, whilst reducing NHS staff data collection burden.Wellbeing Software®, provider of the Euroking® Maternity Information System, added new study specific data fields to their information system, extracted data from participating NHS sites and transferred data for matching with the National Neonatal Research Database to ascertain outcomes for babies admitted to neonatal units. Study set-up processes were put in place for all sites. The data extraction, linkage and cleaning processes were piloted with one pre-selected NHS site.

Results

Twenty-six NHS sites were set-up over 27 months (January 2019 - April 2021). Twenty-four thousand maternity records were extracted from the one NHS site, pertaining to the period January 2015 to March 2019. Data field completeness for maternal and neonatal primary outcomes were mostly acceptable. Neonatal identifiers flowed to the National Neonatal Research Database for successful matching and linkage between maternity and neonatal unit records.

Discussion

Piloting the data extraction and linkage highlighted the need for additional governance arrangements, training at NHS sites and new processes for the study team to ensure data quality and confidentiality are upheld during the study. Amending existing NHS electronic information systems and accessing clinical data at scale, is possible, but continues to be a time consuming and a technically challenging exercise.

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

AIMS AND OBJECTIVES

To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings.

METHOD

PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised.

RESULTS

AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making.

CONCLUSION

AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.

Transforming Clinical Information Systems: Empowering Healthcare through Telemedicine, Data Science, and Artificial Intelligence Applications

OBJECTIVE

In this synopsis, the editors of the Clinical Information Systems (CIS) section of the IMIA Yearbook of Medical Informatics overview recent research and propose a selection of best papers published in 2022 in the CIS field.

METHODS

The editors follow a systematic approach to gather relevant articles and select the best papers for the section. This year, they updated the query to incorporate the topic of telemedicine and removed search terms related to geographic information systems. The revised query resulted in a larger number of identified papers, necessitating the appointment of a third section editor to handle the increased workload. The editors narrowed the initial pool of articles to 15 candidate papers through a multi-stage selection process. At least seven independent reviews were collected for each candidate paper, and a selection meeting with the IMIA Yearbook editorial board led to the final selection of the best papers for the CIS section.

RESULTS

The query was carried out in mid-January 2023 and retrieved a deduplicated result set of 5,206 articles from 1,500 journals. This year, 15 papers were nominated as candidates, and four were finally selected as the best papers in the CIS section.Including telemedicine in the query resulted in a substantial increase in the number of papers found. The analysis highlights the growing convergence between clinical information systems and telemedicine, with mobile health (mHealth) technologies and data science applications gaining prominence. The selected candidate papers emphasize the practical impact of research efforts, focusing on patient-centric outcomes and benefits, including intelligent mobile health monitoring systems and AI-assisted decision-making in healthcare.

CONCLUSIONS

Looking ahead, the field of CIS is expected to continue evolving, driven by advances in telemedicine, mHealth technologies, data science, and AI integration, leading to more efficient, patient-oriented, and intelligent healthcare systems and overall improvement of global healthcare outcomes.

Navigating Infection Control Processes in a COVID-19 Only Safety-Net Hospital at the Height of the Pandemic

Hospitals faced extraordinary challenges during the pandemic. Some of these were directly related to patient care-expanding capacities, adjusting services, and using new knowledge to save lives in a dynamically changing situation. Other challenges were regulatory. The COVID-19 pandemic significantly disrupted routine hospital infection control practices. We report the results of an interview study with 13 individuals associated with infection control in a small independent hospital. We employed the Systems Engineering Initiative for Patient Safety (SEIPS) model as a theoretical framework and as a basis to analyze data. The findings revealed how routine practices and protocols were displaced in notable ways. Due to COVID-19, clinical activities were modified, and the increased demands of regulatory reporting became laborious, and punitive if reports were late. Strategies are needed to mitigate increases in healthcare-associated infections. Our examination of the information flows, transformation, and needs shows areas in which digital tool creation and the use of a trained informatics workforce could ameliorate and automate many processes.

Blockchain-enabled immutable, distributed, and highly available clinical research activity logging system for federated COVID-19 data analysis from multiple institutions

OBJECTIVE

We aimed to develop a distributed, immutable, and highly available cross-cloud blockchain system to facilitate federated data analysis activities among multiple institutions.

MATERIALS AND METHODS

We preprocessed 9166 COVID-19 Structured Query Language (SQL) code, summary statistics, and user activity logs, from the GitHub repository of the Reliable Response Data Discovery for COVID-19 (R2D2) Consortium. The repository collected local summary statistics from participating institutions and aggregated the global result to a COVID-19-related clinical query, previously posted by clinicians on a website. We developed both on-chain and off-chain components to store/query these activity logs and their associated queries/results on a blockchain for immutability, transparency, and high availability of research communication. We measured run-time efficiency of contract deployment, network transactions, and confirmed the accuracy of recorded logs compared to a centralized baseline solution.

RESULTS

The smart contract deployment took 4.5 s on an average. The time to record an activity log on blockchain was slightly over 2 s, versus 5-9 s for baseline. For querying, each query took on an average less than 0.4 s on blockchain, versus around 2.1 s for baseline.

DISCUSSION

The low deployment, recording, and querying times confirm the feasibility of our cross-cloud, blockchain-based federated data analysis system. We have yet to evaluate the system on a larger network with multiple nodes per cloud, to consider how to accommodate a surge in activities, and to investigate methods to lower querying time as the blockchain grows.

CONCLUSION

Blockchain technology can be used to support federated data analysis among multiple institutions.

Programming techniques for improving rule readability for rule-based information extraction natural language processing pipelines of unstructured and semi-structured medical texts

BACKGROUND

Extraction of medical terms and their corresponding values from semi-structured and unstructured texts of medical reports can be a time-consuming and error-prone process. Methods of natural language processing (NLP) can help define an extraction pipeline for accomplishing a structured format transformation strategy.

OBJECTIVES

In this paper, we build an NLP pipeline to extract values of the classification of malignant tumors (TNM) from unstructured and semi-structured pathology reports and import them further to a structured data source for a clinical study. Our research interest is not focused on standard performance metrics like precision, recall, and F-measure on the test and validation data. We discuss how with the help of software programming techniques the readability of rule-based (RB) information extraction (IE) pipelines can be improved, and therefore minimize the time to correct or update the rules, and efficiently import them to another programming language.

METHODS

The extract rules were manually programmed with training data of TNM classification and tested in two separate pipelines based on design specifications from domain experts and data curators. Firstly we implemented each rule directly in one line for each extraction item. Secondly, we reprogrammed them in a readable fashion through decomposition and intention-revealing names for the variable declaration. To measure the impact of both methods we measure the time for the fine-tuning and programming of the extractions through test data of semi-structured and unstructured texts.

RESULTS

We analyze the benefits of improving through readability of the writing of rules, through parallel programming with regular expressions (REGEX), and the Apache Uima Ruta language (AURL). The time for correcting the readable rules in AURL and REGEX was significantly reduced. Complicated rules in REGEX are decomposed and intention-revealing declarations were reprogrammed in AURL in 5 min.

CONCLUSION

We discuss the importance of factor readability and how can it be improved when programming RB text IE pipelines. Independent of the features of the programming language and the tools applied, a readable coding strategy can be proven beneficial for future maintenance and offer an interpretable solution for understanding the extraction and for transferring the rules to other domains and NLP pipelines.

Clinical information system (CIS) implementation in developing countries: requirements, success factors, and recommendations

OBJECTIVE

Clinical Information System (CIS) usage can reduce healthcare costs over time, improve the quality of medical care and safety, and enhance clinical efficiency. However, CIS implementation in developing countries poses additional, different challenges from the developed countries. Therefore, this research aimed to systematically review the literature, gathering and integrating research findings on Success Factors (SFs) in CIS implementation for developing countries. This helps to integrate past knowledge and develop a set of recommendations, presented as a framework, for implementing CIS in developing countries.

MATERIALS AND METHODS

A systematic literature review was conducted, followed by qualitative data analysis on the published articles related to requirements and SF for CIS implementation. Eighty-three articles met the inclusion criteria and were included in the data analysis. Thematic analysis and cross-case analysis were applied to identify and categorize the requirements and SF for CIS implementation in developing countries.

RESULTS

Six major requirement categories were identified including project management, financial resources, government involvement and support, human resources, organizational, and technical requirements. Subcategories related to SF are classified under each major requirement. A set of recommendations is provided, presented in a framework, based on the project management lifecycle approach.

CONCLUSION

The proposed framework could support CIS implementations in developing countries while enhancing their rate of success. Future studies should focus on identifying barriers to CIS implementation in developing countries. The country-specific empirical studies should also be conducted based on this research's findings to match the local context.

"All Quiet on the Western Front" - Clinical Information Systems Research in the Year 2021

OBJECTIVES

In this synopsis, we give an overview of recent research and propose a selection of best papers published in 2021 in the field of Clinical Information Systems (CIS).

METHOD

As CIS section editors, we annually apply a systematic process to retrieve articles for the IMIA Yearbook of Medical Informatics. For eight years now, we use the same query to find relevant publications in the CIS field. Each year we retrieve more than 2,400 papers which we categorize in a multi-pass review to distill a preselection of up to 15 candidate papers. External reviewers and yearbook editors then assess the selected candidate papers. Based on the review results, the IMIA Yearbook editorial board chooses up to four best publications for the section at a selection meeting. To get a comprehensive overview of the content of the retrieved articles, we use text mining and term co-occurrence mapping techniques.

RESULTS

We carried out the query in mid-January 2022 and retrieved a deduplicated result set of 2,688 articles from 1,062 different journals. This year, we nominated ten papers as candidates and finally selected two of them as the best papers in the CIS section. As in the previous years, the content analysis of the articles revealed the broad spectrum of topics covered by CIS research, but - on the other side - no real innovations or new upcoming research trends. However, the significant impact of COVID-19 on CIS research was observable also this year.

CONCLUSIONS

The trends in CIS research, as seen in recent years, continue to be observable. The content analysis revealed nothing really new in the CIS domain. What was very visible was the impact of the COVID-19 pandemic, which still effects our lives and also CIS.

Patient Recruitment System for Clinical Trials: Mixed Methods Study About Requirements at Ten University Hospitals

BACKGROUND

Clinical trials are the gold standard for advancing medical knowledge and improving patient outcomes. For their success, an appropriately sized cohort is required. However, patient recruitment remains one of the most challenging aspects of clinical trials. Information technology (IT) support systems-for instance, patient recruitment systems-may help overcome existing challenges and improve recruitment rates, when customized to the user needs and environment.

OBJECTIVE

The goal of our study is to describe the status quo of patient recruitment processes and to identify user requirements for the development of a patient recruitment system.

METHODS

We conducted a web-based survey with 56 participants as well as semistructured interviews with 33 participants from 10 German university hospitals.

RESULTS

We here report the recruitment procedures and challenges of 10 university hospitals. The recruitment process was influenced by diverse factors such as the ward, use of software, and the study inclusion criteria. Overall, clinical staff seemed more involved in patient identification, while the research staff focused on screening tasks. Ad hoc and planned screenings were common. Identifying eligible patients was still associated with significant manual efforts. The recruitment staff used Microsoft Office suite because tailored software were not available. To implement such software, data from disparate sources will need to be made available. We discussed concrete technical challenges concerning patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration, and we contributed to the support of developing a successful system.

CONCLUSIONS

Identifying eligible patients is still associated with significant manual efforts. To fully make use of the high potential of IT in patient recruitment, many technical and process challenges have to be solved first. We contribute and discuss concrete technical challenges for patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration.

The Clinical Information Systems Response to the COVID-19 Pandemic

OBJECTIVE

The year 2020 was predominated by the coronavirus disease 2019 (COVID-19) pandemic. The objective of this article is to review the areas in which clinical information systems (CIS) can be and have been utilized to support and enhance the response of healthcare systems to pandemics, focusing on COVID-19.

METHODS

PubMed/MEDLINE, Google Scholar, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies pertaining to CIS, pandemics, and COVID-19 through October 2020. The most informative and detailed studies were highlighted, while many others were referenced.

RESULTS

CIS were heavily relied upon by health systems and governmental agencies worldwide in response to COVID-19. Technology-based screening tools were developed to assist rapid case identification and appropriate triaging. Clinical care was supported by utilizing the electronic health record (EHR) to onboard frontline providers to new protocols, offer clinical decision support, and improve systems for diagnostic testing. Telehealth became the most rapidly adopted medical trend in recent history and an essential strategy for allowing safe and effective access to medical care. Artificial intelligence and machine learning algorithms were developed to enhance screening, diagnostic imaging, and predictive analytics - though evidence of improved outcomes remains limited. Geographic information systems and big data enabled real-time dashboards vital for epidemic monitoring, hospital preparedness strategies, and health policy decision making. Digital contact tracing systems were implemented to assist a labor-intensive task with the aim of curbing transmission. Large scale data sharing, effective health information exchange, and interoperability of EHRs remain challenges for the informatics community with immense clinical and academic potential. CIS must be used in combination with engaged stakeholders and operational change management in order to meaningfully improve patient outcomes.

CONCLUSION

Managing a pandemic requires widespread, timely, and effective distribution of reliable information. In the past year, CIS and informaticists made prominent and influential contributions in the global response to the COVID-19 pandemic.

EXpectation Propagation LOgistic REgRession on permissioned blockCHAIN (ExplorerChain): decentralized online healthcare/genomics predictive model learning

Objective

Predicting patient outcomes using healthcare/genomics data is an increasingly popular/important area. However, some diseases are rare and require data from multiple institutions to construct generalizable models. To address institutional data protection policies, many distributed methods keep the data locally but rely on a central server for coordination, which introduces risks such as a single point of failure. We focus on providing an alternative based on a decentralized approach. We introduce the idea using blockchain technology for this purpose, with a brief description of its own potential advantages/disadvantages.

Materials and Methods

We explain how our proposed EXpectation Propagation LOgistic REgRession on Permissioned blockCHAIN (ExplorerChain) can achieve the same results when compared to a distributed model that uses a central server on 3 healthcare/genomic datasets, and what trade-offs need to be considered when using centralized/decentralized methods. We explain how the use of blockchain technology can help decrease some of the problems encountered in decentralized methods.

Results

We showed that the discrimination power of ExplorerChain can be statistically similar to its counterpart central server-based algorithm. While ExplorerChain inherited some benefits of blockchain, it had a small increased running time.

Discussion

ExplorerChain has the same prerequisites as a distributed model with a centralized server for coordination. In a manner similar to secure multi-party computation strategies, it assumes that participating institutions are honest, but “curious.”

Conclusion

When evaluated on relatively small datasets, results suggest that ExplorerChain, which combines artificial intelligence and blockchain technologies, performs as well as a central server-based method, and may avoid some risks at the cost of efficiency.

Privacy-preserving model learning on a blockchain network-of-networks

Objective

To facilitate clinical/genomic/biomedical research, constructing generalizable predictive models using cross-institutional methods while protecting privacy is imperative. However, state-of-the-art methods assume a “flattened” topology, while real-world research networks may consist of “network-of-networks” which can imply practical issues including training on small data for rare diseases/conditions, prioritizing locally trained models, and maintaining models for each level of the hierarchy. In this study, we focus on developing a hierarchical approach to inherit the benefits of the privacy-preserving methods, retain the advantages of adopting blockchain, and address practical concerns on a research network-of-networks.

Materials and Methods

We propose a framework to combine level-wise model learning, blockchain-based model dissemination, and a novel hierarchical consensus algorithm for model ensemble. We developed an example implementation HierarchicalChain (hierarchical privacy-preserving modeling on blockchain), evaluated it on 3 healthcare/genomic datasets, as well as compared its predictive correctness, learning iteration, and execution time with a state-of-the-art method designed for flattened network topology.

Results

HierarchicalChain improves the predictive correctness for small training datasets and provides comparable correctness results with the competing method with higher learning iteration and similar per-iteration execution time, inherits the benefits of the privacy-preserving learning and advantages of blockchain technology, and immutable records models for each level.

Discussion

HierarchicalChain is independent of the core privacy-preserving learning method, as well as of the underlying blockchain platform. Further studies are warranted for various types of network topology, complex data, and privacy concerns.

Conclusion

We demonstrated the potential of utilizing the information from the hierarchical network-of-networks topology to improve prediction.

Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals

OBJECTIVE

To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches.

MATERIALS AND METHODS

Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included.

RESULTS

Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate.

CONCLUSIONS

A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

The anatomy of a distributed predictive modeling framework: online learning, blockchain network, and consensus algorithm

Objective

Cross-institutional distributed healthcare/genomic predictive modeling is an emerging technology that fulfills both the need of building a more generalizable model and of protecting patient data by only exchanging the models but not the patient data. In this article, the implementation details are presented for one specific blockchain-based approach, ExplorerChain, from a software development perspective. The healthcare/genomic use cases of myocardial infarction, cancer biomarker, and length of hospitalization after surgery are also described.

Materials and Methods

ExplorerChain’s 3 main technical components, including online machine learning, metadata of transaction, and the Proof-of-Information-Timed (PoINT) algorithm, are introduced in this study. Specifically, the 3 algorithms (ie, core, new network, and new site/data) are described in detail.

Results

ExplorerChain was implemented and the design details of it were illustrated, especially the development configurations in a practical setting. Also, the system architecture and programming languages are introduced. The code was also released in an open source repository available at https://github.com/tsungtingkuo/explorerchain.

Discussion

The designing considerations of semi-trust assumption, data format normalization, and non-determinism was discussed. The limitations of the implementation include fixed-number participating sites, limited join-or-leave capability during initialization, advanced privacy technology yet to be included, and further investigation in ethical, legal, and social implications.

Conclusion

This study can serve as a reference for the researchers who would like to implement and even deploy blockchain technology. Furthermore, the off-the-shelf software can also serve as a cornerstone to accelerate the development and investigation of future healthcare/genomic blockchain studies.

G3DMS: Design and Implementation of a Data Management System for the Diagnosis of Genetic Disorders

Current health information systems used in genetic research centers and clinics in the Kingdom of Saudi Arabia have failed to enable researchers and health care physicians to utilize genetic and clinical data in their research. In this paper, we aim to design and implement a Genetic Disorders Diagnosis Data Management System (G3DMS) to support clinicians in the process of diagnosing genetic diseases and conducting genetic studies. A case study was undertaken to analyze a health information system in Saudi to understand its design problems via a brainstorming method. We then used the Barker’s system design method and a prototype to validate our proposed system via usability testing. This research has resulted in the development of the G3DMS that comprises: electronic data-capture forms for data entry; a customized query builder to display and modify patient data as well as form research queries; a module that allows historical data to be uploaded in the form of bulk data using a template; export data options to Excel and JavaScript Object Notation (JSON) format; and authorization access for healthcare researchers and clinicians. The G3DMS was implemented in the Princess Al-Jawhara Center of Excellence in Research of Hereditary Disorders, Jeddah, KSA.

Adapting Clinical Systems to Enable Adolescents' Genomic Choices

BACKGROUND

We offered adolescents personalized choices about the type of genetic results they wanted to learn during a research study and created a workflow to filter and transfer the results to the electronic health record (EHR).

METHODS

We describe adaptations needed to ensure that adolescents' results documented in the EHR and returned to adolescent/parent dyads matched their choices. A web application enabled manual modification of the underlying laboratory report data based on adolescents' choices. The final PDF format of the laboratory reports was not viewable through the EHR patient portal, so an EHR form was created to support the manual entry of discrete results that could be viewed in the portal.

RESULTS

Enabling adolescents' choices about genetic results was a labor-intensive process. More than 350 hours was required for development of the application and EHR form, as well as over 50 hours of a study professional's time to enter choices into the application and EHR. Adolescents and their parents who learned genetic results through the patient portal indicated that they were satisfied with the method of return and would make their choices again if given the option.

CONCLUSION

Although future EHR upgrades are expected to enable patient portal access to PDFs, additional improvements are needed to allow the results to be partitioned and filtered based on patient preferences. Furthermore, separating these results into more discrete components will allow them to be stored separately in the EHR, supporting the use of these data in clinical decision support or artificial intelligence applications.

Applied Practice and Possible Leverage Points for Information Technology Support for Patient Screening in Clinical Trials: Qualitative Study

BACKGROUND

Clinical trials are one of the most challenging and meaningful designs in medical research. One essential step before starting a clinical trial is screening, that is, to identify patients who fulfill the inclusion criteria and do not fulfill the exclusion criteria. The screening step for clinical trials might be supported by modern information technology (IT).

OBJECTIVE

This explorative study aimed (1) to obtain insights into which tools for feasibility estimations and patient screening are actually used in clinical routine and (2) to determine which method and type of IT support could benefit clinical staff.

METHODS

Semistandardized interviews were conducted in 5 wards (cardiology, gynecology, gastroenterology, nephrology, and palliative care) in a German university hospital. Of the 5 interviewees, 4 were directly involved in patient screening. Three of them were clinicians, 1 was a study nurse, and 1 was a research assistant.

RESULTS

The existing state of study feasibility estimation and the screening procedure were dominated by human communication and estimations from memory, although there were many possibilities for IT support. Success mostly depended on the experience and personal motivation of the clinical staff. Electronic support has been used but with little importance so far. Searches in ward-specific patient registers (databases) and searches in clinical information systems were reported. Furthermore, free-text searches in medical reports were mentioned. For potential future applications, a preference for either proactive or passive systems was not expressed. Most of the interviewees saw the potential for the improvement of the actual systems, but they were also largely satisfied with the outcomes of the current approach. Most of the interviewees were interested in learning more about the various ways in which IT could support and relieve them in their clinical routine.

CONCLUSIONS

Overall, IT support currently plays a minor role in the screening step for clinical trials. The lack of IT usage and the estimations made from memory reported by all the participants might constrain cognitive resources, which might distract from clinical routine. We conclude that electronic support for the screening step for clinical trials is still a challenge and that education of the staff about the possibilities for electronic support in clinical trials is necessary.

Fair compute loads enabled by blockchain: sharing models by alternating client and server roles

Objective

Decentralized privacy-preserving predictive modeling enables multiple institutions to learn a more generalizable model on healthcare or genomic data by sharing the partially trained models instead of patient-level data, while avoiding risks such as single point of control. State-of-the-art blockchain-based methods remove the “server” role but can be less accurate than models that rely on a server. Therefore, we aim at developing a general model sharing framework to preserve predictive correctness, mitigate the risks of a centralized architecture, and compute the models in a fair way

Materials and Methods

We propose a framework that includes both server and “client” roles to preserve correctness. We adopt a blockchain network to obtain the benefits of decentralization, by alternating the roles for each site to ensure computational fairness. Also, we developed GloreChain (Grid Binary LOgistic REgression on Permissioned BlockChain) as a concrete example, and compared it to a centralized algorithm on 3 healthcare or genomic datasets to evaluate predictive correctness, number of learning iterations and execution time

Results

GloreChain performs exactly the same as the centralized method in terms of correctness and number of iterations. It inherits the advantages of blockchain, at the cost of increased time to reach a consensus model

Discussion

Our framework is general or flexible and can also address intrinsic challenges of blockchain networks. Further investigations will focus on higher-dimensional datasets, additional use cases, privacy-preserving quality concerns, and ethical, legal, and social implications

Conclusions

Our framework provides a promising potential for institutions to learn a predictive model based on healthcare or genomic data in a privacy-preserving and decentralized way.

Usage of query-based health information exchange after event notifications

Objectives

This study sought to quantify the association between event notifications and subsequent query-based health information exchange (HIE) use among end users of three different community health information organizations.

Materials and Methods

Using system-log data merged with user characteristics, regression-adjusted estimates were used to describe the association between event notifications and subsequent query-based HIE usage.

Results

Approximately 5% of event notifications were associated with query-based HIE usage within 30 days. In adjusted models, odds of query-based HIE usage following an event notification were higher for older patients and for alerts triggered by a discharge event. Query-based HIE usage was more common among specialty clinics and Federally Qualified Health Centers than primary care organizations.

Discussion and Conclusion

In this novel combination of data, 1 in 20 event notifications resulted in subsequent query-based HIE usage. Results from this study suggest that event notifications and query-based HIE can be applied together to address clinical and population health use cases.

Clinical Information Systems and Artificial Intelligence: Recent Research Trends

OBJECTIVES

This survey aims at reviewing the literature related to Clinical Information Systems (CIS), Hospital Information Systems (HIS), Electronic Health Record (EHR) systems, and how collected data can be analyzed by Artificial Intelligence (AI) techniques.

METHODS

We selected the major journals (11 journals) collecting papers (more than 7,000) over the last five years from the top members of the research community, and read and analyzed the papers (more than 200) covering the topics. Then, we completed the analysis using search engines to also include papers from major conferences over the same five years.

RESULTS

We defined a taxonomy of major features and research areas of CIS, HIS, EHR systems. We also defined a taxonomy for the use of Artificial Intelligence (AI) techniques on healthcare data. In the light of these taxonomies, we report on the most relevant papers from the literature.

CONCLUSIONS

We highlighted some major research directions and issues which seem to be promising and to need further investigations over a medium- or long-term period.

Connecting the clinical IT infrastructure to a service-oriented architecture of medical devices

The new medical device communication protocol known as IEEE 11073 SDC is well-suited for the integration of (surgical) point-of-care devices, so are the established Health Level Seven (HL7) V2 and Digital Imaging and Communications in Medicine (DICOM) standards for the communication of systems in the clinical IT infrastructure (CITI). An integrated operating room (OR) and other integrated clinical environments, however, need interoperability between both domains to fully unfold their potential for improving the quality of care as well as clinical workflows. This work thus presents concepts for the propagation of clinical and administrative data to medical devices, physiologic measurements and device parameters to clinical IT systems, as well as image and multimedia content in both directions. Prototypical implementations of the derived components have proven to integrate well with systems of networked medical devices and with the CITI, effectively connecting these heterogeneous domains. Our qualitative evaluation indicates that the interoperability concepts are suitable to be integrated into clinical workflows and are expected to benefit patients and clinicians alike. The upcoming HL7 Fast Healthcare Interoperability Resources (FHIR) communication standard will likely change the domain of clinical IT significantly. A straightforward mapping to its resource model thus ensures the tenability of these concepts despite a foreseeable change in demand and requirements.

Team Competencies and Educational Threshold Concepts for Clinical Information Modelling

Healthcare interoperability depends upon sound semantic models to support safe and reliable exchange of information. We argue that clinical information modelling requires a collaborative team of healthcare professionals, process and content analysts and terminologists and that 'separation of concerns' is unhelpful. We present six fundamental concepts that participants must understand to collaborate meaningfully in technology-agnostic information modelling.

An observational study of the relationship between meaningful use-based electronic health information exchange, interoperability, and medication reconciliation capabilities

Stagnation in hospitals' adoption of data integration functionalities coupled with reduction in the number of operational health information exchanges could become a significant impediment to hospitals' adoption of 3 critical capabilities: electronic health information exchange, interoperability, and medication reconciliation, in which electronic systems are used to assist with resolving medication discrepancies and improving patient safety. Against this backdrop, we assessed the relationships between the 3 capabilities.We conducted an observational study applying partial least squares-structural equation modeling technique to 27 variables obtained from the 2013 American Hospital Association annual survey Information Technology (IT) supplement, which describes health IT capabilities.We included 1330 hospitals. In confirmatory factor analysis, out of the 27 variables, 15 achieved loading values greater than 0.548 at P < .001, as such were validated as the building blocks of the 3 capabilities. Subsequent path analysis showed a significant, positive, and cyclic relationship between the capabilities, in that decreases in the hospitals' adoption of one would lead to decreases in the adoption of the others.These results show that capability for high quality medication reconciliation may be impeded by lagging adoption of interoperability and health information exchange capabilities. Policies focused on improving one or more of these capabilities may have ancillary benefits.

Enhancing Patient Safety Event Reporting. A Systematic Review of System Design Features

OBJECTIVES

Electronic patient safety event reporting (e-reporting) is an effective mechanism to learn from errors and enhance patient safety. Unfortunately, the value of e-reporting system (a software or web server based platform) in patient safety research is greatly overshadowed by low quality reporting. This paper aims at revealing the current status of system features, detecting potential gaps in system design, and accordingly proposing suggestions for future design and implementation of the system.

METHODS

Three literature databases were searched for publications that contain informative descriptions of e-reporting systems. In addition, both online publicly accessible reporting forms and systems were investigated.

RESULTS

48 systems were identified and reviewed. 11 system design features and their frequencies of occurrence (Top 5: widgets (41), anonymity or confidentiality (29), hierarchy (20), validator (17), review notification (15)) were identified and summarized into a system hierarchical model.

CONCLUSIONS

The model indicated the current e-reporting systems are at an immature stage in their development, and discussed their future development direction toward efficient and effective systems to improve patient safety.

Electronic Alerts Improve Immunization Rates in Two-month-old Premature Infants Hospitalized in the Neonatal Intensive Care Unit

OBJECTIVE

To determine if an electronic alert improves 2 month immunization rates in infants remaining hospitalized in the neonatal intensive care unit.

METHODS

Institutional Review Board-approved retrospective chart review of 261 infants with birth weights <2 kg and still hospitalized at ≥ 58 days. Charts were reviewed between 2009 and 2013, before and after the 2011 electronic alert was instituted in the electronic medical record from days 56 to 67 to remind providers that immunizations were due. Order and administration dates of two-month vaccine components (Diphtheria, Haemophilus influenza B, Hepatitis B, Pertussis, Pneumococcal, Polio, Tetanus) were determined, and infants were considered fully immunized, partially immunized, or unimmunized by day 90 or discharge, whichever came first.

RESULTS

After the alert, the timing of vaccine orders decreased from day 67 to day 61 (p<0.0001) and vaccine administration decreased from day 71 to day 64 (p<0.0001). Missing vaccine orders decreased from 14% [17/121] to 3% [4/140] (p=0.001) with missing administrations decreasing from 21% [26/121] to 4% [6/140] (p<0.0001). Fully immunized rates increased from 71% [86/121] to 94% [132/140] (p<0.0001).

CONCLUSIONS

A significant improvement in immunization rates in two-month-old infants in the neonatal intensive care unit occurred by 90 days after implementing an alert in the electronic medical record.

Findings from the Clinical Information Systems Perspective for 2015

OBJECTIVE

To summarize recent research and to propose a selection of best papers published in 2015 in the field of Clinical Information Systems (CIS).

METHOD

The query which had been used last year to retrieve articles for the CIS section of the IMIA Yearbook of Medical Informatics 2015 was refined. It again aimed at identifying relevant publications in the field of CIS and comprised search terms from the Medical Subject Headings (MeSH) catalog as well as additional free text search terms from PubMed and Web of Science. The retrieved articles were categorized in a multi-pass review carried out separately by the two section editors. The final selection of 15 candidate papers was then peer-reviewed by Yearbook editors and external reviewers. Based on the review results the four best papers were then selected at the best papers selection meeting with the IMIA Yearbook editorial board. To get an overview on the content of the retrieved articles we applied text mining and term co-occurrence mapping techniques.

RESULTS

The query was carried out in mid-January 2016, yielding a combined result set of 1851 articles which were published in 790 different journals. The most relevant terms from abstracts and titles of these articles were assigned to six different clusters. A majority of articles dealt with two thematic blocks, problems and solutions in the CIS field. The majority of the 2016 CIS candidate papers and all four best papers could be assigned to these two thematic blocks.

CONCLUSIONS

We identified two main tracks among the CIS candidate and best papers as well as in CIS research activities in general: problems and solutions. A never ending cycle of continuous improvement.

WebDISCO: a web service for distributed cox model learning without patient-level data sharing

OBJECTIVE

The Cox proportional hazards model is a widely used method for analyzing survival data. To achieve sufficient statistical power in a survival analysis, it usually requires a large amount of data. Data sharing across institutions could be a potential workaround for providing this added power.

METHODS AND MATERIALS

The authors develop a web service for distributed Cox model learning (WebDISCO), which focuses on the proof-of-concept and algorithm development for federated survival analysis. The sensitive patient-level data can be processed locally and only the less-sensitive intermediate statistics are exchanged to build a global Cox model. Mathematical derivation shows that the proposed distributed algorithm is identical to the centralized Cox model.

RESULTS

The authors evaluated the proposed framework at the University of California, San Diego (UCSD), Emory, and Duke. The experimental results show that both distributed and centralized models result in near-identical model coefficients with differences in the range [Formula: see text] to [Formula: see text]. The results confirm the mathematical derivation and show that the implementation of the distributed model can achieve the same results as the centralized implementation.

LIMITATION

The proposed method serves as a proof of concept, in which a publicly available dataset was used to evaluate the performance. The authors do not intend to suggest that this method can resolve policy and engineering issues related to the federated use of institutional data, but they should serve as evidence of the technical feasibility of the proposed approach.Conclusions WebDISCO (Web-based Distributed Cox Regression Model; https://webdisco.ucsd-dbmi.org:8443/cox/) provides a proof-of-concept web service that implements a distributed algorithm to conduct distributed survival analysis without sharing patient level data.

Findings from the Clinical Information Systems Perspective

OBJECTIVE

To summarize recent research and to propose a selection of best papers published in 2014 in the field of Clinical Information Systems (CIS).

METHOD

A query with search terms from the Medical Subject Headings (MeSH) catalog as well as additional free text search terms was designed to identify relevant publications in the field of clinical information systems from PubMed and Web of Science®. The retrieved articles were then categorized in a multi-pass review carried out separately by the section editors. The final selection of 15 candidate papers was then peerreviewed by Yearbook editors and external reviewers. Based on the review results the four best papers were then selected at the best papers selection meeting with the IMIA Yearbook editorial board.

RESULTS

The query was carried out in mid-January 2015, yielding a combined result set of 1525 articles which were published in 722 different journals. Among these articles two main thematic sections were identified: i) Interoperability from a syntactical and semantic point of view as well as from a longterm preservation and organizational/legal point of view and ii) secondary use of existing health data in all its shades. Here, patient safety was a major scope of application.

CONCLUSIONS

CIS have become mature over the last years. The focus has now moved beyond data acquisition for just supporting the local care workflows. Actual research efforts in the CIS domain comprise the breakdown of information silos, the reduction of barriers between different systems of different care providers and secondary use of accumulated health data for multiple purposes.

CohortExplorer: A Generic Application Programming Interface for Entity Attribute Value Database Schemas

BACKGROUND

Most electronic data capture (EDC) and electronic data management (EDM) systems developed to collect and store clinical data from participants recruited into studies are based on generic entity-attribute-value (EAV) database schemas which enable rapid and flexible deployment in a range of study designs. The drawback to such schemas is that they are cumbersome to query with structured query language (SQL). The problem increases when researchers involved in multiple studies use multiple electronic data capture and management systems each with variation on the EAV schema.

OBJECTIVE

The aim of this study is to develop a generic application which allows easy and rapid exploration of data and metadata stored under EAV schemas that are organized into a survey format (questionnaires/events, questions, values), in other words, the Clinical Data Interchange Standards Consortium (CDISC) Observational Data Model (ODM).

METHODS

CohortExplorer is written in Perl programming language and uses the concept of SQL abstract which allows the SQL query to be treated like a hash (key-value pairs).

RESULTS

We have developed a tool, CohortExplorer, which once configured for a EAV system will "plug-n-play" with EAV schemas, enabling the easy construction of complex queries through an abstracted interface. To demonstrate the utility of the CohortExplorer system, we show how it can be used with the popular EAV based frameworks; Opal (OBiBa) and REDCap.

CONCLUSIONS

The application is available under a GPL-3+ license at the CPAN website. Currently the application only provides datasource application programming interfaces (APIs) for Opal and REDCap. In the future the application will be available with datasource APIs for all major electronic data capture and management systems such as OpenClinica and LabKey. At present the application is only compatible with EAV systems where the metadata is organized into surveys, questionnaires and events. Further work is needed to make the application compatible with EAV schemas where the metadata is organized into hierarchies such as Informatics for Integrating Biology & the Bedside (i2b2). A video tutorial demonstrating the application setup, datasource configuration, and search features is available on YouTube. The application source code is available at the GitHub website and the users are encouraged to suggest new features and contribute to the development of APIs for new EAV systems.

Impact of homecare electronic health record on timeliness of clinical documentation, reimbursement, and patient outcomes

BACKGROUND

Homecare is an important and effective way of managing chronic illnesses using skilled nursing care in the home. Unlike hospitals and ambulatory settings, clinicians visit patients at home at different times, independent of each other. Twenty-nine percent of 10,000 homecare agencies in the United States have adopted point-of-care EHRs. Yet, relatively little is known about the growing use of homecare EHRs.

OBJECTIVE

Researchers compared workflow, financial billing, and patient outcomes before and after implementation to evaluate the impact of a homecare point-of-care EHR.

METHODS

The design was a pre/post observational study embedded in a mixed methods study. The setting was a Philadelphia-based homecare agency with 137 clinicians. Data sources included: (1) clinician EHR documentation completion; (2) EHR usage data; (3) Medicare billing data; (4) an EHR Nurse Satisfaction survey; (5) clinician observations; (6) clinician interviews; and (7) patient outcomes.

RESULTS

Clinicians were satisfied with documentation timeliness and team communication. Following EHR implementation, 90% of notes were completed within the 1-day compliance interval (n = 56,702) compared with 30% of notes completed within the 7-day compliance interval in the pre-implementation period (n = 14,563; OR 19, p <. 001). Productivity in the number of clinical notes documented post-implementation increased almost 10-fold compared to pre-implementation. Days to Medicare claims fell from 100 days pre-implementation to 30 days post-implementation, while the census rose. EHR implementation impact on patient outcomes was limited to some behavioral outcomes.

DISCUSSION

Findings from this homecare EHR study indicated clinician EHR use enabled a sustained increase in productivity of note completion, as well as timeliness of documentation and billing for reimbursement with limited impact on improving patient outcomes. As EHR adoption increases to better meet the needs of the growing population of older people with chronic health conditions, these results can inform homecare EHR development and implementation.

Challenges and facilitators to adoption of a point-of-care electronic health record in home care

Electronic health records (EHRs), intended to improve the clinical process, are understudied in home care. The researchers assessed clinician satisfaction, informed by workflow and patient outcomes, to identify EHR adoption challenges. The mixed methods study setting was a Philadelphia agency with 137 clinicians. Adoption challenges included: (a) hardware problems coupled with lack of field support; (b) inadequate training; and (c) mismatch of EHR usability/functionality and workflow resulting in decreased efficiency. Adoption facilitators were support for team communication and improved clinical data timeliness. Opportunities for improved adoption included sharing with front-line clinicians EHR data related to patient care and health outcomes.

Clinical information transfer and data capture in the acute myocardial infarction pathway: an observational study

RATIONALE, AIMS AND OBJECTIVES

Acute myocardial infarctions (MIs) or heart attacks are the result of a complete or an incomplete occlusion of the lumen of the coronary artery with a thrombus. Prompt diagnosis and early coronary intervention results in maximum myocardial salvage, hence time to treat is of the essence. Adequate, accurate and complete information is vital during the early stages of admission of an MI patient and can impact significantly on the quality and safety of patient care. This study aimed to record how clinical information between different clinical teams during the journey of a patient in the MI care pathway is captured and to review the flow of information within this care pathway.

METHOD

A prospective, descriptive, structured observational study to assess (i) current clinical information systems (CIS) utilization and (ii) real-time information availability within an acute cardiac care setting was carried out. Completeness and availability of patient information capture across four key stages of the MI care pathway were assessed prospectively.

RESULTS

Thirteen separate information systems were utilized during the four phases of the MI pathway. Observations revealed fragmented CIS utilization, with users accessing an average of six systems to gain a complete set of patient information. Data capture was found to vary between each pathway stage and in both patient cohort risk groupings. The highest level of information completeness (100%) was observed only in the discharge stage of the MI care pathway. The lowest level of information completeness (58%) was observed in the admission stage.

CONCLUSION

The study highlights fragmentation, CIS duplication, and discrepancies in the current clinical information capture and data transfer across the MI care pathway in an acute cardiac care setting. The development of an integrated and user-friendly electronic data capture and transfer system would reduce duplication and would facilitate efficient and complete information provision at the point of care.

Health care transformation through collaboration on open-source informatics projects: integrating a medical applications platform, research data repository, and patient summarization

Background

The Strategic Health IT Advanced Research Projects (SHARP) program seeks to conquer well-understood challenges in medical informatics through breakthrough research. Two SHARP centers have found alignment in their methodological needs: (1) members of the National Center for Cognitive Informatics and Decision-making (NCCD) have developed knowledge bases to support problem-oriented summarizations of patient data, and (2) Substitutable Medical Apps, Reusable Technologies (SMART), which is a platform for reusable medical apps that can run on participating platforms connected to various electronic health records (EHR). Combining the work of these two centers will ensure wide dissemination of new methods for synthesized views of patient data. Informatics for Integrating Biology and the Bedside (i2b2) is an NIH-funded clinical research data repository platform in use at over 100 sites worldwide. By also working with a co-occurring initiative to SMART-enabling i2b2, we can confidently write one app that can be used extremely broadly.

Objective

Our goal was to facilitate development of intuitive, problem-oriented views of the patient record using NCCD knowledge bases that would run in any EHR. To do this, we developed a collaboration between the two SHARPs and an NIH center, i2b2.

Methods

First, we implemented collaborative tools to connect researchers at three institutions. Next, we developed a patient summarization app using the SMART platform and a previously validated NCCD problem-medication linkage knowledge base derived from the National Drug File-Reference Terminology (NDF-RT). Finally, to SMART-enable i2b2, we implemented two new Web service “cells” that expose the SMART application programming interface (API), and we made changes to the Web interface of i2b2 to host a “carousel” of SMART apps.

Results

We deployed our SMART-based, NDF-RT-derived patient summarization app in this SMART-i2b2 container. It displays a problem-oriented view of medications and presents a line-graph display of laboratory results.

Conclusions

This summarization app can be run in any EHR environment that either supports SMART or runs SMART-enabled i2b2. This i2b2 “clinical bridge” demonstrates a pathway for reusable app development that does not require EHR vendors to immediately adopt the SMART API. Apps can be developed in SMART and run by clinicians in the i2b2 repository, reusing clinical data extracted from EHRs. This may encourage the adoption of SMART by supporting SMART app development until EHRs adopt the platform. It also allows a new variety of clinical SMART apps, fueled by the broad aggregation of data types available in research repositories. The app (including its knowledge base) and SMART-i2b2 are open-source and freely available for download.

Chronic Care Model Decision Support and Clinical Information Systems interventions for people living with HIV: a systematic review

BACKGROUND

The Chronic Care Model is an effective framework for improving chronic disease management. There is scarce literature describing this model for people living with HIV. Decision Support (DS) and Clinical Information Systems (CIS) are two components of this model that aim to improve care by changing health care provider behavior.

OBJECTIVE

Our aim was to assess the effectiveness of DS and CIS interventions for individuals with HIV, through a systematic literature review.

DESIGN

We performed systematic electronic searches from 1996 to February 2011 of the medical (E.g. Medline, EMBASE, CINAHL) and grey literature. Effectiveness was measured by the frequency of statistically significant outcome improvement. Data and key equity indicator extraction and synthesis was completed.

PARTICIPANTS AND INTERVENTIONS

We included comparative studies of people living with HIV that examined the impact of DS or CIS interventions on outcomes.

MAIN MEASURES

The following measures were assessed: outcome (immunological/virological, medical, psychosocial, economic measures) and health care process/performance measures.

KEY RESULTS

Records were screened for relevance (n = 10,169), full-text copies of relevant studies were obtained (n = 123), and 16 studies were included in the review. Overall, 5/9 (55.6%) and 17/41 (41.5%) process measures and 5/12 (41.7%) and 3/9 (33.3%) outcome measures for DS and CIS interventions, respectively, were statistically significantly improved. DS-explicit mention of implementation of guidelines and CIS-reminders showed the most frequent improvement in outcomes. DS-only interventions were more effective than CIS-only interventions in improving both process and outcome measures. Clinical, statistical and methodological heterogeneity among studies precluded meta-analysis. Primary studies were methodologically weak and often included multifaceted interventions that made assessment of effectiveness challenging.

CONCLUSIONS

Overall, DS and CIS interventions may modestly improve care for people living with HIV, having a greater impact on process measures compared to outcome measures. These interventions should be considered as part of strategies to improve HIV care through changing provider performance.

Interdisciplinary care team adoption of electronic point-of-care documentation systems: an unrealized opportunity

We conducted three health care evaluation studies in community and hospital settings to examine adoption of point-of-care documentation systems among interdisciplinary care team clinicians. Both community studies used a mixed methods design to assess actual system usage and clinician satisfaction. In the hospitals, scenario testing was used. Results indicated clinician adoption of the systems was universal, although not always timely with: (1) a mismatch between system functionality and workflow which was a barrier to clinician system access during patient care and reduced clinician efficiency; (2) no increase in interdisciplinary team communication; and (3) no impact on patient outcomes identified by clinicians. To facilitate adoption, clinicians should see the value of using the system as intended by receiving patient care and patient safety feedback that uses system data.

Development and evaluation of a crowdsourcing methodology for knowledge base construction: identifying relationships between clinical problems and medications

OBJECTIVE

We describe a novel, crowdsourcing method for generating a knowledge base of problem-medication pairs that takes advantage of manually asserted links between medications and problems.

METHODS

Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem-medication links for inclusion in a knowledge base that can be used to infer previously unasserted links between problems and medications.

RESULTS

Clinicians manually linked 231,223 medications (55.30% of prescribed medications) to problems within the electronic health record, generating 41,203 distinct problem-medication pairs, although not all were accurate. We developed methods to evaluate the accuracy of the pairs, and after limiting the pairs to those meeting an estimated 95% appropriateness threshold, 11,166 pairs remained. The pairs in the knowledge base accounted for 183,127 total links asserted (76.47% of all links). Retrospective application of the knowledge base linked 68,316 medications not previously linked by a clinician to an indicated problem (36.53% of unlinked medications). Expert review of the combined knowledge base, including inferred and manually linked problem-medication pairs, found a sensitivity of 65.8% and a specificity of 97.9%.

CONCLUSION

Crowdsourcing is an effective, inexpensive method for generating a knowledge base of problem-medication pairs that is automatically mapped to local terminologies, up-to-date, and reflective of local prescribing practices and trends.

An integrated EHR at Northern California Kaiser Permanente: pitfalls, challenges, and benefits experienced in transitioning

As legacy information systems age, transition to newer systems is inevitable, but at times fraught with challenge. This brief article addresses some of the pitfalls, challenges, and benefits we experienced at Kaiser Permanente as we transitioned several key clinical information systems to Epic Systems for our integrated comprehensive Electronic Health Record (EHR).

A new Web-based medical tool for assessment and prevention of comprehensive cardiovascular risk

Background:

Multifactor cardiovascular disease is the leading cause of death; besides well-known cardiovascular risk factors, several emerging factors such as mental stress, diet type, and physical inactivity, have been associated to cardiovascular disease. To date, preventive strategies are based on the concept of absolute risk calculated by different algorithms and scoring systems. However, in general practice the patient’s data collection represents a critical issue.

Design:

A new multipurpose computer-based program has been developed in order to:1) easily calculate and compare the absolute cardiovascular risk by the Framingham, Procam, and Progetto Cuore algorithms; 2) to design a web-based computerized tool for prospective collection of structured data; 3) to support the doctor in the decision-making process for patients at risk according to recent international guidelines.

Methods:

During a medical consultation the doctor utilizes a common computer connected by Internet to a medical server where all the patient’s data and software reside. The program evaluates absolute and relative cardiovascular risk factors, personalized patient’s goals, and multiparametric trends, monitors critical parameter values, and generates an automated medical report.

Results:

In a pilot study on 294 patients (47% males; mean age 60 ± 12 years [±SD]) the global time to collect data at first consultation was 13 ± 11 minutes which declined to 8 ± 7 minutes at the subsequent consultation. In 48.2% of cases the program revealed 2 or more primary risk factor parameters outside guideline indications and gave specific clinical suggestions to return altered parameters to target values.

Conclusion:

The web-based system proposed here may represent a feasible and flexible tool for clinical management of patients at risk of cardiovascular disease and for epidemiological research.

Advice for Decision Makers Based on an Electronic Health Record Evaluation at a Program for All-inclusive Care for Elders Site

OBJECTIVE

Provide evidence-based advise to "Program of All-inclusive Care for the Elderly" (PACE) decision makers considering implementing an electronic health record (EHR) system, drawing on the results of a mixed methods study to examine: (1) the diffusion of an EHR among clinicians documenting direct patient care in a PACE day care site, (2) the impact of the use of the EHR on the satisfaction levels of clinicians, and (3) the impact of the use of the EHR on patient functional outcomes.

METHODS

Embedded mixed methods design with a post-test design quantitative experiment and concurrent qualitative component. Quantitative methods included: (1) the EHR audit log used to determine the frequency and timing during the week of clinicians' usage of the system; (2) a 22-item clinician satisfaction survey; and (3) a 16-item patient functional outcome questionnaire related to locomotion, mobility, personal hygiene, dressing, feeding as well the use of adaptive devices. Qualitative methods included observations and open-ended, semi-structured follow-up interviews. Qualitative data was merged with the quantitative data by comparing the findings along themes. The setting was a PACE utilizing an EHR in Philadelphia: PACE manages the care of nursing-home eligible members to enable them to avoid nursing home admission and reside in their homes. Participants were 39 clinicians on the multi-disciplinary teams caring for the elders and 338 PACE members.

RESULTS

Clinicians did not use the system as intended, which may help to explain why the benefits related to clinical processes and patient outcomes as expected for an EHR were not reflected in the results. Clinicians were satisfied with the EHR, although there was a non-significant decline between 11 and 17 months post implementation of the EHR. There was no significant difference in patient functional outcome the two time periods. However, the sample size of 48 was too small to allow any conclusive statements to be made. Interpretation of findings underscores the importance of the interaction of workflow and EHR functionality and usability to impact clinician satisfaction, efficiency, and clinician use of the EHR.

CONCLUSION

This research provides insights into EHR use in the care of the older people in community-based health care settings. This study assessed the adoption of an EHR outside the acute hospital setting and in the community setting to provide evidence-based recommendations to PACE decision makers considering implementing an EHR.

Confronting and resolving an ethical dilemma associated with a practice based evaluation using observational methodology of health information technology

SUMMARY

As the adoption of health information technology (HIT) has escalated, efforts to evaluate its uptake have increased. The evaluation of HIT often requires direct observation of health care practitioners interacting with the system. When in the field, the evaluator who is not a trained health care provider may observe suboptimal use of the technology. If evaluators have plans to share the results of the evaluation at the conclusion of the study, they face a decision point about whether to disclose interim results and the implications of doing so. To provide HIT evaluators with guidance about what issues to weigh when observing the implementation of HIT, this paper presents a study of an actual case and discusses the following considerations: (1) whether the evaluation of HIT is considered to be human subject research; (2) if the evaluation is human subject research, whether the Institutional Review Board will consider it exempt from review or subjected to expedited or full review; and (3) how interim disclosure to the clinic management impacts the research study. The recommendations to evaluators include use of a protocol for interim disclosures to patients, clinicians, and/or clinical management for both quality assurance initiatives and human subjects research.

Results of a statewide evaluation of "paperwork burden" in addiction treatment

This article chronicles three steps taken by research, clinical, and state staff toward assessing, evaluating, and streamlining clinical and administrative paperwork at all public outpatient addiction treatment programs in one state. The first step was an accounting of all paperwork requirements at each program. The second step included the development of time estimates for the paperwork requirements; synthesis of information across sites; providing written evaluation of the need, utility, and redundancy of all forms (paperwork) collected; and suggestions for eliminating unused or unnecessary data collection and streamlining the remaining data collection. Thirdly, the state agency hosted a meeting with the state staff, researchers, and staff from all programs and agencies with state-funded contracts and took action. Paperwork reductions over the course of a 6-month outpatient treatment episode were estimated at 4 to 6 hours, with most of the time burden being eliminated from the intake process.

Information technology to support improved care for chronic illness

BACKGROUND

In populations with chronic illness, outcomes improve with the use of care models that integrate clinical information, evidence-based treatments, and proactive management of care. Health information technology is believed to be critical for efficient implementation of these chronic care models. Health care organizations have implemented information technologies, such as electronic medical records, to varying degrees. However, considerable uncertainty remains regarding the relative impact of specific informatics technologies on chronic illness care.

OBJECTIVE

To summarize knowledge and increase expert consensus regarding informatics components that support improvement in chronic illness care.

DESIGN

A systematic review of the literature was performed. "Use case" models were then developed, based on the literature review, and guidance from clinicians and national quality improvement projects. A national expert panel process was conducted to increase consensus regarding information system components that can be used to improve chronic illness care.

RESULTS

The expert panel agreed that informatics should be patient-centered, focused on improving outcomes, and provide support for illness self-management. They concurred that outcomes should be routinely assessed, provided to clinicians during the clinical encounter, and used for population-based care management. It was recommended that interactive, sequential, disorder-specific treatment pathways be implemented to quickly provide clinicians with patient clinical status, treatment history, and decision support.

CONCLUSIONS

Specific informatics strategies have the potential to improve care for chronic illness. Software to implement these strategies should be developed, and rigorously evaluated within the context of organizational efforts to improve care.