Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study

The Unified Medical Language System at 30 Years and How It Is Used and Published: Systematic Review and Content Analysis

BACKGROUND

The Unified Medical Language System (UMLS) has been a critical tool in biomedical and health informatics, and the year 2021 marks its 30th anniversary. The UMLS brings together many broadly used vocabularies and standards in the biomedical field to facilitate interoperability among different computer systems and applications.

OBJECTIVE

Despite its longevity, there is no comprehensive publication analysis of the use of the UMLS. Thus, this review and analysis is conducted to provide an overview of the UMLS and its use in English-language peer-reviewed publications, with the objective of providing a comprehensive understanding of how the UMLS has been used in English-language peer-reviewed publications over the last 30 years.

METHODS

PubMed, ACM Digital Library, and the Nursing & Allied Health Database were used to search for studies. The primary search strategy was as follows: UMLS was used as a Medical Subject Headings term or a keyword or appeared in the title or abstract. Only English-language publications were considered. The publications were screened first, then coded and categorized iteratively, following the grounded theory. The review process followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

A total of 943 publications were included in the final analysis. Moreover, 32 publications were categorized into 2 categories; hence the total number of publications before duplicates are removed is 975. After analysis and categorization of the publications, UMLS was found to be used in the following emerging themes or areas (the number of publications and their respective percentages are given in parentheses): natural language processing (230/975, 23.6%), information retrieval (125/975, 12.8%), terminology study (90/975, 9.2%), ontology and modeling (80/975, 8.2%), medical subdomains (76/975, 7.8%), other language studies (53/975, 5.4%), artificial intelligence tools and applications (46/975, 4.7%), patient care (35/975, 3.6%), data mining and knowledge discovery (25/975, 2.6%), medical education (20/975, 2.1%), degree-related theses (13/975, 1.3%), digital library (5/975, 0.5%), and the UMLS itself (150/975, 15.4%), as well as the UMLS for other purposes (27/975, 2.8%).

CONCLUSIONS

The UMLS has been used successfully in patient care, medical education, digital libraries, and software development, as originally planned, as well as in degree-related theses, the building of artificial intelligence tools, data mining and knowledge discovery, foundational work in methodology, and middle layers that may lead to advanced products. Natural language processing, the UMLS itself, and information retrieval are the 3 most common themes that emerged among the included publications. The results, although largely related to academia, demonstrate that UMLS achieves its intended uses successfully, in addition to achieving uses broadly beyond its original intentions.

Pragmatic MDR: a metadata repository with bottom-up standardization of medical metadata through reuse

Background

The variety of medical documentation often leads to incompatible data elements that impede data integration between institutions. A common approach to standardize and distribute metadata definitions are ISO/IEC 11179 norm-compliant metadata repositories with top-down standardization. To the best of our knowledge, however, it is not yet common practice to reuse the content of publicly accessible metadata repositories for creation of case report forms or routine documentation. We suggest an alternative concept called pragmatic metadata repository, which enables a community-driven bottom-up approach for agreeing on data collection models. A pragmatic metadata repository collects real-world documentation and considers frequent metadata definitions as high quality with potential for reuse.

Methods

We implemented a pragmatic metadata repository proof of concept application and filled it with medical forms from the Portal of Medical Data Models. We applied this prototype in two use cases to demonstrate its capabilities for reusing metadata: first, integration into a study editor for the suggestion of data elements and, second, metadata synchronization between two institutions. Moreover, we evaluated the emergence of bottom-up standards in the prototype and two medical data managers assessed their quality for 24 medical concepts.

Results

The resulting prototype contained 466,569 unique metadata definitions. Integration into the study editor led to a reuse of 1836 items and item groups. During the metadata synchronization, semantic codes of 4608 data elements were transferred. Our evaluation revealed that for less complex medical concepts weak bottom-up standards could be established. However, more diverse disease-related concepts showed no convergence of data elements due to an enormous heterogeneity of metadata. The survey showed fair agreement (K_alpha = 0.50, 95% CI 0.43–0.56) for good item quality of bottom-up standards.

Conclusions

We demonstrated the feasibility of the pragmatic metadata repository concept for medical documentation. Applications of the prototype in two use cases suggest that it facilitates the reuse of data elements. Our evaluation showed that bottom-up standardization based on a large collection of real-world metadata can yield useful results. The proposed concept shall not replace existing top-down approaches, rather it complements them by showing what is commonly used in the community to guide other researchers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01524-8.

Artificial Intelligence in Medicine: Chances and Challenges for Wide Clinical Adoption

Background

Artificial intelligence (AI) applications that utilize machine learning are on the rise in clinical research and provide highly promising applications in specific use cases. However, wide clinical adoption remains far off. This review reflects on common barriers and current solution approaches.

Summary

Key challenges are abbreviated as the RISE criteria: Regulatory aspects, Interpretability, interoperability, and the need for Structured data and Evidence. As reoccurring barriers of AI adoption, these concepts are delineated and complemented by points to consider and possible solutions for effective and safe use of AI applications.

Key Messages

There is a fraction of AI applications with proven clinical benefits and regulatory approval. Many new promising systems are the subject of current research but share common issues for wide clinical adoption. The RISE criteria can support preparation for challenges and pitfalls when designing or introducing AI applications into clinical practice.

Conceptual Design, Implementation, and Evaluation of Generic and Standard-Compliant Data Transfer into Electronic Health Records

Objectives  The objective of this study is the conceptual design, implementation and evaluation of a system for generic, standard-compliant data transfer into electronic health records (EHRs). This includes patient data from clinical research and medical care that has been semantically annotated and enhanced with metadata. The implementation is based on the single-source approach. Technical and clinical feasibilities, as well as cost-benefit efficiency, were investigated in everyday clinical practice.

Methods  Münster University Hospital is a tertiary care hospital with 1,457 beds and 10,823 staff who treated 548,110 patients in 2018. Single-source metadata architecture transformation (SMA:T) was implemented as an extension to the EHR system. This architecture uses Model Driven Software Development (MDSD) to generate documentation forms according to the Clinical Data Interchange Standards Consortium (CDISC) operational data model (ODM). Clinical data are stored in ODM format in the EHR system database. Documentation forms are based on Google's Material Design Standard. SMA:T was used at a total of five clinics and one administrative department in the period from March 1, 2018 until March 31, 2019 in everyday clinical practice.

Results  The technical and clinical feasibility of SMA:T was demonstrated in the course of the study. Seventeen documentation forms including 373 data items were created with SMA:T. Those were created for 2,484 patients by 283 users in everyday clinical practice. A total of 121 documentation forms were examined retrospectively. The Constructive cost model (COCOMO II) was used to calculate cost and time savings. The form development mean time was reduced by 83.4% from 3,357 to 557 hours. Average costs per form went down from EUR 953 to 158.

Conclusion  Automated generic transfer of standard-compliant data and metadata into EHRs is technically and clinically feasible, cost efficient, and a useful method to establish comprehensive and semantically annotated clinical documentation. Savings of time and personnel resources are possible.

Implementation of Mock Acute Care Advance Pharmacy Practice Experience Simulations and an Assessment Rubric

Objective. To implement a mock acute care advanced pharmacy practice experience series into the didactic training of second-year pharmacy students and validate an accompanying assessment rubric. Methods. Three 90-minute acute care patient simulation laboratory sessions were developed with input from clinical specialists, preceptors, students, and faculty members. An accompanying student evaluation rubric was also developed. The assessment rubric was validated using pairs of preceptor raters to determine inter-rater reliability, along with predictive validity on advanced pharmacy practice experience (APPE) acute care scores. A student survey was also conducted. Results. The mock acute care APPEs were successfully implemented into the didactic curriculum. The assessment rubric had good inter-rater reliability and good predictive validity with acute care APPEs. Survey results indicated that students found the mock acute care APPE simulation laboratories useful. Conclusion. Other schools seeking to enhance their students' preparedness for and performance in acute care APPEs should consider implementing acute care APPE simulations in the didactic curriculum.