recruIT: A cloud-native clinical trial recruitment support system based on Health Level 7 Fast Healthcare Interoperability Resources (HL7 FHIR) and the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM)

BACKGROUND

Clinical trials (CTs) are foundational to the advancement of evidence-based medicine and recruiting a sufficient number of participants is one of the crucial steps to their successful conduct. Yet, poor recruitment remains the most frequent reason for premature discontinuation or costly extension of clinical trials.

METHODS

We designed and implemented a novel, open-source software system to support the recruitment process in clinical trials by generating automatic recruitment recommendations. The development is guided by modern, cloud-native design principles and based on Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) as an interoperability standard with the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) being used as a source of patient data. We evaluated the usability using the system usability scale (SUS) after deploying the application for use by study personnel.

RESULTS

The implementation is based on the OMOP CDM as a repository of patient data that is continuously queried for possible trial candidates based on given clinical trial eligibility criteria. A web-based screening list can be used to display the candidates and email notifications about possible new trial participants can be sent automatically. All interactions between services use HL7 FHIR as the communication standard. The system can be installed using standard container technology and supports more sophisticated deployments on Kubernetes clusters. End-users (n = 19) rated the system with a SUS score of 79.9/100.

CONCLUSION

We contribute a novel, open-source implementation to support the patient recruitment process in clinical trials that can be deployed using state-of-the art technologies. According to the SUS score, the system provides good usability.

Evaluation of a Digital Dementia Registry's IT Architecture After a Three-Year Period in Practice: digiDEM Bayern

INTRODUCTION

The project "digiDEM Bayern" aims to set up a registry with long-term follow-up data on people with dementia and their family caregivers. For that purpose an Electronic Data Capture (EDC) system linked with a Participant Management (PM) system has been established. This study evaluates the acceptance and usability of the IT tools supporting all data management processes in order to further improve the system and associated processes.

METHODS

For this purpose we collected the key numbers of the registry, and used the System Usability Scale (SUS) to evaluate the interactions of the data management systems in a wide area.

RESULTS

Thirty-six research partners (RP) and six study team (ST) members completed the anonymous online survey. The EDC system overall reached an average SUS score of 73.42 and the PM system of 77.92.

DISCUSSION

The two systems fulfil their required task and, therefore, simplify the work of the RP in the data collection process and of the ST during the data quality checks.

CONCLUSION

Integrating the used systems is therefore recommended for registry studies in other medical areas.

Effectiveness of IT-supported patient recruitment: study protocol for an interrupted time series study at ten German university hospitals

BACKGROUND

As part of the German Medical Informatics Initiative, the MIRACUM project establishes data integration centers across ten German university hospitals. The embedded MIRACUM Use Case "Alerting in Care - IT Support for Patient Recruitment", aims to support the recruitment into clinical trials by automatically querying the repositories for patients satisfying eligibility criteria and presenting them as screening candidates. The objective of this study is to investigate whether the developed recruitment tool has a positive effect on study recruitment within a multi-center environment by increasing the number of participants. Its secondary objective is the measurement of organizational burden and user satisfaction of the provided IT solution.

METHODS

The study uses an Interrupted Time Series Design with a duration of 15 months. All trials start in the control phase of randomized length with regular recruitment and change to the intervention phase with additional IT support. The intervention consists of the application of a recruitment-support system which uses patient data collected in general care for screening according to specific criteria. The inclusion and exclusion criteria of all selected trials are translated into a machine-readable format using the OHDSI ATLAS tool. All patient data from the data integration centers is regularly checked against these criteria. The primary outcome is the number of participants recruited per trial and week standardized by the targeted number of participants per week and the expected recruitment duration of the specific trial. Secondary outcomes are usability, usefulness, and efficacy of the recruitment support. Sample size calculation based on simple parallel group assumption can demonstrate an effect size of d=0.57 on a significance level of 5% and a power of 80% with a total number of 100 trials (10 per site). Data describing the included trials and the recruitment process is collected at each site. The primary analysis will be conducted using linear mixed models with the actual recruitment number per week and trial standardized by the expected recruitment number per week and trial as the dependent variable.

DISCUSSION

The application of an IT-supported recruitment solution developed in the MIRACUM consortium leads to an increased number of recruited participants in studies at German university hospitals. It supports employees engaged in the recruitment of trial participants and is easy to integrate in their daily work.

Evaluation of Eligibility Criteria Relevance for the Purpose of IT-Supported Trial Recruitment: Descriptive Quantitative Analysis

BACKGROUND

Clinical trials (CTs) are crucial for medical research; however, they frequently fall short of the requisite number of participants who meet all eligibility criteria (EC). A clinical trial recruitment support system (CTRSS) is developed to help identify potential participants by performing a search on a specific data pool. The accuracy of the search results is directly related to the quality of the data used for comparison. Data accessibility can present challenges, making it crucial to identify the necessary data for a CTRSS to query. Prior research has examined the data elements frequently used in CT EC but has not evaluated which criteria are actually used to search for participants. Although all EC must be met to enroll a person in a CT, not all criteria have the same importance when searching for potential participants in an existing data pool, such as an electronic health record, because some of the criteria are only relevant at the time of enrollment.

OBJECTIVE

In this study, we investigated which groups of data elements are relevant in practice for finding suitable participants and whether there are typical elements that are not relevant and can therefore be omitted.

METHODS

We asked trial experts and CTRSS developers to first categorize the EC of their CTs according to data element groups and then to classify them into 1 of 3 categories: necessary, complementary, and irrelevant. In addition, the experts assessed whether a criterion was documented (on paper or digitally) or whether it was information known only to the treating physicians or patients.

RESULTS

We reviewed 82 CTs with 1132 unique EC. Of these 1132 EC, 350 (30.9%) were considered necessary, 224 (19.8%) complementary, and 341 (30.1%) total irrelevant. To identify the most relevant data elements, we introduced the data element relevance index (DERI). This describes the percentage of studies in which the corresponding data element occurs and is also classified as necessary or supplementary. We found that the query of "diagnosis" was relevant for finding participants in 79 (96.3%) of the CTs. This group was followed by "date of birth/age" with a DERI of 85.4% (n=70) and "procedure" with a DERI of 35.4% (n=29).

CONCLUSIONS

The distribution of data element groups in CTs has been heterogeneously described in previous works. Therefore, we recommend identifying the percentage of CTs in which data element groups can be found as a more reliable way to determine the relevance of EC. Only necessary and complementary criteria should be included in this DERI.

Creating a Medication Therapy Observational Research Database from an Electronic Medical Record: Challenges and Data Curation

BACKGROUND

Observational research has shown its potential to complement experimental research and clinical trials by secondary use of treatment data from hospital care processes. It can also be applied to better understand pediatric drug utilization for establishing safer drug therapy. Clinical documentation processes often limit data quality in pediatric medical records requiring data curation steps, which are mostly underestimated.

OBJECTIVES

The objectives of this study were to transform and curate data from a departmental electronic medical record into an observational research database. We particularly aim at identifying data quality problems, illustrating reasons for such problems and describing the systematic data curation process established to create high-quality data for observational research.

METHODS

Data were extracted from an electronic medical record used by four wards of a German university children's hospital from April 2012 to June 2020. A four-step data preparation, mapping, and curation process was established. Data quality of the generated dataset was firstly assessed following an established 3 × 3 Data Quality Assessment guideline and secondly by comparing a sample subset of the database with an existing gold standard.

RESULTS

The generated dataset consists of 770,158 medication dispensations associated with 89,955 different drug exposures from 21,285 clinical encounters. A total of 6,840 different narrative drug therapy descriptions were mapped to 1,139 standard terms for drug exposures. Regarding the quality criterion correctness, the database was consistent and had overall a high agreement with our gold standard.

CONCLUSION

Despite large amounts of freetext descriptions and contextual knowledge implicitly included in the electronic medical record, we were able to identify relevant data quality issues and to establish a semi-automated data curation process leading to a high-quality observational research database. Because of inconsistent dosage information in the original documentation this database is limited to a drug utilization database without detailed dosage information.

Machine learning in medication prescription: A systematic review

BACKGROUND

Medication prescription is a complex process that could benefit from current research and development in machine learning through decision support systems. Particularly pediatricians are forced to prescribe medications "off-label" as children are still underrepresented in clinical studies, which leads to a high risk of an incorrect dose and adverse drug effects.

METHODS

PubMed, IEEE Xplore and PROSPERO were searched for relevant studies that developed and evaluated well-performing machine learning algorithms following the PRISMA statement. Quality assessment was conducted in accordance with the IJMEDI checklist. Identified studies were reviewed in detail, including the required variables for predicting the correct dose, especially of pediatric medication prescription.

RESULTS

The search identified 656 studies, of which 64 were reviewed in detail and 36 met the inclusion criteria. According to the IJMEDI checklist, five studies were considered to be of high quality. 19 of the 36 studies dealt with the active substance warfarin. Overall, machine learning algorithms based on decision trees or regression methods performed superior regarding their predictive power than algorithms based on neural networks, support vector machines or other methods. The use of ensemble methods like bagging or boosting generally enhanced the accuracy of the dose predictions. The required input and output variables of the algorithms were considerably heterogeneous and differ strongly among the respective substance.

CONCLUSIONS

By using machine learning algorithms, the prescription process could be simplified and dosing correctness could be enhanced. Despite the heterogenous results among the different substances and cases and the lack of pediatric use cases, the identified approaches and required variables can serve as an excellent starting point for further development of algorithms predicting drug doses, particularly for children. Especially the combination of physiologically-based pharmacokinetic models with machine learning algorithms represents a great opportunity to enhance the predictive power and accuracy of the developed algorithms.

Towards a Bavarian Oncology Real World Data Research Platform

INTRODUCTION

In the last decade numerous real-world data networks have been established in order to leverage the value of data from electronic health records for medical research. In Germany, a nation-wide network based on electronic health record data from all German university hospitals has been established within the Medical Informatics Initiative (MII) and recently opened for researcherst' access through the German Portal for Medical Research Data (FDPG). In Bavaria, the six university hospitals have joined forces within the Bavarian Cancer Research Center (BZKF). The oncology departments aim at establishing a federated observational research network based on the framework of the MII/FDPG and extending it with a clear focus on oncological clinical data, imaging data and molecular high throughput analysis data.

METHODS

We describe necessary adaptions and extensions of existing MII components with the goal of establishing a Bavarian oncology real world data research platform with its first use case of performing federated feasibility queries on clinical oncology data.

RESULTS

We share insights from developing a feasibility platform prototype and presenting it to end users. Our main discovery was that oncological data is characterized by a higher degree of interdependence and complexity compared to the MII core dataset that is already integrated into the FDPG.

DISCUSSION

The significance of our work lies in the requirements we formulated for extending already existing MII components to match oncology specific data and to meet oncology researchers needs while simultaneously transferring back our results and experiences into further developments within the MII.

A pipeline for the fully automated estimation of continuous reference intervals using real-world data

Reference intervals are essential for interpreting laboratory test results. Continuous reference intervals precisely capture physiological age-specific dynamics that occur throughout life, and thus have the potential to improve clinical decision-making. However, established approaches for estimating continuous reference intervals require samples from healthy individuals, and are therefore substantially restricted. Indirect methods operating on routine measurements enable the estimation of one-dimensional reference intervals, however, no automated approach exists that integrates the dependency on a continuous covariate like age. We propose an integrated pipeline for the fully automated estimation of continuous reference intervals expressed as a generalized additive model for location, scale and shape based on discrete model estimates using an indirect method (refineR). The results are free of subjective user-input, enable conversion of test results into z-scores and can be integrated into laboratory information systems. Comparison of our results to established and validated reference intervals from the CALIPER and PEDREF studies and manufacturers' package inserts shows good agreement of reference limits, indicating that the proposed pipeline generates high-quality results. In conclusion, the developed pipeline enables the generation of high-precision percentile charts and continuous reference intervals. It represents the first parameter-less and fully automated solution for the indirect estimation of continuous reference intervals.

Towards a National Portal for Medical Research Data (FDPG): Vision, Status, and Lessons Learned

Harmonizing medical data sharing frameworks is challenging. Data collection and formats follow local solutions in individual hospitals; thus, interoperability is not guaranteed. The German Medical Informatics Initiative (MII) aims to provide a Germany-wide, federated, large-scale data sharing network. In the last five years, numerous efforts have been successfully completed to implement the regulatory framework and software components for securely interacting with decentralized and centralized data sharing processes. 31 German university hospitals have today established local data integration centers that are connected to the central German Portal for Medical Research Data (FDPG). Here, we present milestones and associated major achievements of various MII working groups and subprojects which led to the current status. Further, we describe major obstacles and the lessons learned during its routine application in the last six months.

Reproducibility in 2023 - An End-to-End Template for Analysis and Manuscript Writing

Reproducibility imposes some special requirements at different stages of each project, including reproducible workflows for the analysis including to follow best practices regarding code style and to make the creation of the manuscript reproducible as well. Available tools therefore include version control systems such as Git and document creation tools such as Quarto or R Markdown. However, a re-usable project template mapping the entire process from performing the data analysis to finally writing the manuscript in a reproducible manner is yet lacking. This work aims to fill this gap by presenting an open source template for conducting reproducible research projects utilizing a containerized framework for both developing and conducting the analysis and summarizing the results in a manuscript. This template can be used instantly without any customization.

Does Voluntary Work Contribute to Cognitive Performance? - An International Systematic Review

Introduction

There is a need for knowledge on activities that can reduce cognitive decline and dementia risk. Volunteering is a productive activity that entails social, physical, and cognitive functions. Therefore, volunteering could be a protective factor for cognitive loss. Thus, this review aims to examine the associations between volunteering and volunteers' cognition and to identify influencing variables.

Methods

Six international literature databases were searched for relevant articles published between 2017 and 2021 (ALOIS, CENTRAL, CINAL, Embase, PsycINFO, PubMed). Quantitative studies of all study designs were included. The primary outcome was the volunteers' cognition measured by objective, internationally established psychometric function tests. Two authors independently assessed the eligibility and quality of the studies. A narrative synthesis was performed using all studies included in this review. The methodology was in line with the PRISMA guidelines.

Results

Fourteen studies met the inclusion criteria and were included. Seven of the included studies confirmed that volunteering positively affects the volunteers' cognitive function. Two other studies identified an association between volunteer activity and volunteers' cognition using cross-sectional measurements. In particular, women and people with a low level of education benefit from the positive effects and associations. The study quality of the included articles was moderate to weak.

Discussion

Our review suggests that volunteering can improve volunteers' cognition. Unfortunately, little attention is given to specific volunteer activities and the frequency of engagement. Additionally, more attention is needed on various risk factors of cognitive impairment.

A FHIR has been lit on gICS: facilitating the standardised exchange of informed consent in a large network of university medicine

BACKGROUND

The Federal Ministry of Education and Research of Germany (BMBF) funds a network of university medicines (NUM) to support COVID-19 and pandemic research at national level. The "COVID-19 Data Exchange Platform" (CODEX) as part of NUM establishes a harmonised infrastructure that supports research use of COVID-19 datasets. The broad consent (BC) of the Medical Informatics Initiative (MII) is agreed by all German federal states and forms the legal base for data processing. All 34 participating university hospitals (NUM sites) work upon a harmonised infrastructural as well as legal basis for their data protection-compliant collection and transfer of their research dataset to the central CODEX platform. Each NUM site ensures that the exchanged consent information conforms to the already-balloted HL7 FHIR consent profiles and the interoperability concept of the MII Task Force "Consent Implementation" (TFCI). The Independent Trusted Third-Party (TTP) of the University Medicine Greifswald supports data protection-compliant data processing and provides the consent management solutions gICS.

METHODS

Based on a stakeholder dialogue a required set of FHIR-functionalities was identified and technically specified supported by official FHIR experts. Next, a "TTP-FHIR Gateway" for the HL7 FHIR-compliant exchange of consent information using gICS was implemented. A last step included external integration tests and the development of a pre-configured consent template for the BC for the NUM sites.

RESULTS

A FHIR-compliant gICS-release and a corresponding consent template for the BC were provided to all NUM sites in June 2021. All FHIR functionalities comply with the already-balloted FHIR consent profiles of the HL7 Working Group Consent Management. The consent template simplifies the technical BC rollout and the corresponding implementation of the TFCI interoperability concept at the NUM sites.

CONCLUSIONS

This article shows that a HL7 FHIR-compliant and interoperable nationwide exchange of consent information could be built using of the consent management software gICS and the provided TTP-FHIR Gateway. The initial functional scope of the solution covers the requirements identified in the NUM-CODEX setting. The semantic correctness of these functionalities was validated by project-partners from the Ludwig-Maximilian University in Munich. The production rollout of the solution package to all NUM sites has started successfully.

The effects of exercise programs on cognition, activities of daily living, and neuropsychiatric symptoms in community-dwelling people with dementia—a systematic review

Background

The prevalence of dementia is expected to increase dramatically. Due to a lack of pharmacological treatment options for people with dementia, non-pharmacological treatments such as exercise programs have been recommended to improve cognition, activities of daily living, and neuropsychiatric symptoms. However, inconsistent results have been reported across different trials, mainly because of the high heterogeneity of exercise modalities. Thus, this systematic review aims to answer the questions whether exercise programs improve cognition, activities of daily living as well as neuropsychiatric symptoms in community-dwelling people with dementia.

Methods

Eight databases were searched for articles published between 2016 and 2021 (ALOIS, CENTRAL, CINAHL, Embase, MEDLINE, PsycINFO, PubMed, Web of Science). Randomized controlled trials evaluating the effects of any type of physical activity on cognition, activities of daily living, or neuropsychiatric symptoms in community-dwelling people with a formal diagnosis of dementia were included in this systematic review. Two authors independently assessed eligibility and quality of the studies. The methodology was in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines.

Results

Eight publications covering seven trials were included in this review with the majority investigating either a combination of strength and aerobic exercise or aerobic exercise alone. This review revealed that there is no clear evidence for the beneficial effects of exercise on cognition. None of the included trials found an impact on activities of daily living. Although different randomized controlled trials reported inconsistent results, one trial indicated that especially aerobic exercise may improve neuropsychiatric symptoms.

Conclusion

Our systematic review did not confirm the impact of exercise on cognition and activities of daily living in community-dwelling people with dementia. The results suggested that aerobic exercise might be effective to reduce neuropsychiatric symptoms. Well-designed trials including only community-dwelling people with a formal diagnosis of dementia, large samples, long-term follow-ups, and detailed description of adherence to the intervention are needed to improve the scientific evidence on the best type of exercise modality.

Trial registration

PROSPERO, CRD42021246598.

Empowering researchers to query medical data and biospecimens by ensuring appropriate usability: Evaluation study of the ABIDE_MI feasibility tool (Preprint)

BACKGROUND

The Aligning Biobanking and Data Integration Centers Efficiently project aims to harmonize technologies and governance structures of German university hospitals and their biobanks to facilitate searching for patient data and biospecimens. The central element will be a feasibility tool for researchers to query the availability of samples and data to determine the feasibility of their study project.

OBJECTIVE

The objectives of the study were as follows: an evaluation of the overall user interface usability of the feasibility tool, the identification of critical usability issues, comprehensibility of the underlying ontology operability, and analysis of user feedback on additional functionalities. From these, recommendations for quality-of-use optimization, focusing on more intuitive usability, were derived.

METHODS

To achieve the study goal, an exploratory usability test consisting of 2 main parts was conducted. In the first part, the thinking aloud method (test participants express their thoughts aloud throughout their use of the tool) was complemented by a quantitative questionnaire. In the second part, the interview method was combined with supplementary mock-ups to collect users' opinions on possible additional features.

RESULTS

The study cohort rated global usability of the feasibility tool based on the System Usability Scale with a good score of 81.25. The tasks assigned posed certain challenges. No participant was able to solve all tasks correctly. A detailed analysis showed that this was mostly because of minor issues. This impression was confirmed by the recorded statements, which described the tool as intuitive and user friendly. The feedback also provided useful insights regarding which critical usability problems occur and need to be addressed promptly.

CONCLUSIONS

The findings indicate that the prototype of the Aligning Biobanking and Data Integration Centers Efficiently feasibility tool is headed in the right direction. Nevertheless, we see potential for optimization primarily in the display of the search functions, the unambiguous distinguishability of criteria, and the visibility of their associated classification system. Overall, it can be stated that the combination of different tools used to evaluate the feasibility tool provided a comprehensive picture of its usability.

RIbench: A Proposed Benchmark for the Standardized Evaluation of Indirect Methods for Reference Interval Estimation

BACKGROUND

Indirect methods leverage real-world data for the estimation of reference intervals. These constitute an active field of research, and several methods have been developed recently. So far, no standardized tool for evaluation and comparison of indirect methods exists.

METHODS

We provide RIbench, a benchmarking suite for quantitative evaluation of any existing or novel indirect method. The benchmark contains simulated test sets for 10 biomarkers mimicking routine measurements of a mixed distribution of non-pathological (reference) values and pathological values. The non-pathological distributions represent 4 common distribution types: normal, skewed, heavily skewed, and skewed-and-shifted. To identify strengths and weaknesses of indirect methods, test sets have varying sample sizes and pathological distributions differ in location, extent of overlap, and fraction. For performance evaluation, we use an overall benchmark score and sub-scores derived from absolute z-score deviations between estimated and true reference limits. We illustrate the application of RIbench by evaluating and comparing the Hoffmann method and 4 modern indirect methods -TML (Truncated-Maximum-Likelihood), kosmic, TMC (Truncated-Minimum-Chi-Square), and refineR- against one another and against a nonparametric direct method (n = 120).

RESULTS

For the modern indirect methods, pathological fraction and sample size had a strong influence on the results: With a pathological fraction up to 20% and a minimum sample size of 5000, most methods achieved results comparable or superior to the direct method.

CONCLUSIONS

We present RIbench, an open-source R-package, for the systematic evaluation of existing and novel indirect methods. RIbench can serve as a tool for enhancement of indirect methods, improving the estimation of reference intervals.

DQAgui: a graphical user interface for the MIRACUM data quality assessment tool

Background

With the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.

Methods

The aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.’s DQ categories conformance, completeness and plausibility.

Results

With DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.

Conclusions

As part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01961-z.

AHD2FHIR: A Tool for Mapping of Natural Language Annotations to Fast Healthcare Interoperability Resources - A Technical Case Report

A significant portion of data in Electronic Health Records is only available as unstructured text, such as surgical or finding reports, clinical notes and discharge summaries. To use this data for secondary purposes, natural language processing (NLP) tools are required to extract structured information. Furthermore, for interoperable use, harmonization of the data is necessary. HL7 Fast Healthcare Interoperability Resources (FHIR), an emerging standard for exchanging healthcare data, defines such a structured format. For German-language medical NLP, the tool Averbis Health Discovery (AHD) represents a comprehensive solution. AHD offers a proprietary REST interface for text analysis pipelines. To build a bridge between FHIR and this interface, we created a service that translates the communication around AHD from and to FHIR. The application is available under an open source license.

Availability of Structured Data Elements in Electronic Health Records for Supporting Patient Recruitment in Clinical Trials

Automated identification of eligible patients for clinical trials is an evident secondary use for electronic health records (EHR) data accumulated during routine care. This task requires relevant data elements to be both available in the EHR and in a structured form. This work analyzes these data quality dimensions of EHR data elements corresponding to a selection of frequent eligibility criteria over a total of 436 patient records at 10 university hospitals within the MIRACUM consortium. Data elements from demographics, diagnosis and laboratory findings are typically structured with a completeness of 73 % to 88 % while medication as well as procedures are rather untructured with a completeness of only 44 %. The results can be used to derive suggestions for data quality improvement measures with respect to patient recruitment as well as to serve as a baseline to quantify future developments.

Attitudes and Acceptance Towards Artificial Intelligence in Medical Care

BACKGROUND

Artificial intelligence (AI) in medicine is a very topical issue. As far as the attitudes and perspectives of the different stakeholders in healthcare are concerned, there is still much to be explored.

OBJECTIVE

Our aim was to determine attitudes and aspects towards acceptance of AI applications from the perspective of physicians in university hospitals.

METHODS

We conducted individual exploratory expert interviews. Low fidelity mockups were used to show interviewees potential application areas of AI in clinical care.

RESULTS

In principle, physicians are open to the use of AI in medical care. However, they are critical of some aspects such as data protection or the lack of explainability of the systems.

CONCLUSION

Although some trends in attitudes e.g., on the challenges or benefits of using AI became clear, it is necessary to conduct further research as intended by the subsequent PEAK project.

The COVID-19 Data Exchange Platform of the German University Medicine

COVID-19 has challenged the healthcare systems worldwide. To quickly identify successful diagnostic and therapeutic approaches large data sharing approaches are inevitable. Though organizational clinical data are abundant, many of them are available only in isolated silos and largely inaccessible to external researchers. To overcome and tackle this challenge the university medicine network (comprising all 36 German university hospitals) has been founded in April 2020 to coordinate COVID-19 action plans, diagnostic and therapeutic strategies and collaborative research activities. 13 projects were initiated from which the CODEX project, aiming at the development of a Germany-wide Covid-19 Data Exchange Platform, is presented in this publication. We illustrate the conceptual design, the stepwise development and deployment, first results and the current status.

Classification of Patient Portals Described in Evaluation Studies Using the TOPCOP Taxonomy

Many patient portals have been introduced and evaluated in recent years. The results of evaluation studies are difficult to compare, however, as the evaluated patient portal is often not clearly or only incompletely described in the publication. This problem is common to evaluations in health informatics. We evaluated the completeness of descriptions of patient portals in 15 exemplary evaluation publications using the TOPCOP taxonomy. Our results show that core functionalities such as portal design, patient communication, educational features, or system notifications were quite clearly described in all 15 evaluation studies. Other descriptions, such as web accessibility or data management, were often not provided. We conclude that taxonomies such as TOPCOP should be used and even required for describing interventions in evaluation papers.

Towards a FHIR-Based Data Model for Coronary Angiography Observations

Coronary heart disease is among the most frequent causes of death globally. Thus, our research project aims to develop prognostic models, to predict the risk of spontaneous myocardial infarctions based on a combination of clinical parameters and image data sets (invasive coronary angiograms). To train such models we use data from more than 30,000 coronary angiograms acquired at the cardiology department of Erlangen University Hospital. To linking such proprietary data with additional clinical parameters and to harmonize it for future cross-hospital federated machine learning approaches we defined a mapping for coronary angiography based on the symptom/ clinical phenotype HL7® FHIR® module of the German medical informatics initiative. In this paper we describe the final design of the coronary angiography information model and our mapping approach to ICD-10 and SNOMED CT. From the database we use a subset of 15 required values patient characteristics to create the HL7® FHIR® resource.

Aligning Biobanks and Data Integration Centers Efficiently (ABIDE_MI)

ABIDE_MI is a complementary funded 18 months project within the German Medical Informatics Initiative (MII), which aims to align IT infrastructures and regulatory/governance structures between biobanks/biobanking IT and the MII data integration centres (DIC) at German university hospitals. A major task in 2021 was the systematic collection of all documents describing rules, as well as proposal/contract templates for data and biosample use and access at each of the participating 24 university hospitals and their comparison with MII-wide consented data sharing principles, documents and governance structures. This comparison revealed large heterogeneity across the ABIDE_MI sites and further, redundant structures/regulations currently established at the German university hospitals. A second task was the design and stepwise development of an IT network infrastructure with central components (data and biosample query portal) and decentralized standardized FHIR servers to capture the standardized FHIR-based core data set modules (resources) defined within the MII working group "Interoperability". Subsequent steps in the project are the harmonization of the data and biosample sharing governance/regulation frameworks at each ABIDE_MI site, creating synergies for the research infrastructures at the German university hospitals and to link those resources to the German Portal for Medical Research Data and with the BBMRI-ERIC Directory and Negotiator tools.

Improving COVID-19 Research of University Hospitals in Germany: Formative Usability Evaluation of the CODEX Feasibility Portal

BACKGROUND

 Within the German "Network University Medicine," a portal is to be developed to enable researchers to query on novel coronavirus disease 2019 (COVID-19) data from university hospitals for assessing the feasibility of a clinical study.

OBJECTIVES

 The usability of a prototype for federated feasibility queries was evaluated to identify design strengths and weaknesses and derive improvement recommendations for further development.

METHODS

 In the course of a remote usability test with the thinking-aloud method and posttask interviews, 15 clinical researchers evaluated the usability of a prototype of the Feasibility Portal. The identified usability problems were rated according to severity, and improvement recommendations were derived.

RESULTS

 The design of the prototype was rated as simple, intuitive, and as usable with little effort. The usability test reported a total of 26 problems, 8 of these were rated as "critical." Usability problems and revision recommendations focus primarily on improving the visual distinguishability of selected inclusion and exclusion criteria, enabling a flexible approach to criteria linking, and enhancing the free-text search.

CONCLUSION

 Improvement proposals were developed for these user problems which will guide further development and the adaptation of the portal to user needs. This is an important prerequisite for correct and efficient use in everyday clinical work in the future. Results can provide developers of similar systems with a good starting point for interface conceptualizations. The methodological approach/the developed test guideline can serve as a template for similar evaluations.

Patient Recruitment System for Clinical Trials: Mixed Methods Study About Requirements at Ten University Hospitals

BACKGROUND

Clinical trials are the gold standard for advancing medical knowledge and improving patient outcomes. For their success, an appropriately sized cohort is required. However, patient recruitment remains one of the most challenging aspects of clinical trials. Information technology (IT) support systems-for instance, patient recruitment systems-may help overcome existing challenges and improve recruitment rates, when customized to the user needs and environment.

OBJECTIVE

The goal of our study is to describe the status quo of patient recruitment processes and to identify user requirements for the development of a patient recruitment system.

METHODS

We conducted a web-based survey with 56 participants as well as semistructured interviews with 33 participants from 10 German university hospitals.

RESULTS

We here report the recruitment procedures and challenges of 10 university hospitals. The recruitment process was influenced by diverse factors such as the ward, use of software, and the study inclusion criteria. Overall, clinical staff seemed more involved in patient identification, while the research staff focused on screening tasks. Ad hoc and planned screenings were common. Identifying eligible patients was still associated with significant manual efforts. The recruitment staff used Microsoft Office suite because tailored software were not available. To implement such software, data from disparate sources will need to be made available. We discussed concrete technical challenges concerning patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration, and we contributed to the support of developing a successful system.

CONCLUSIONS

Identifying eligible patients is still associated with significant manual efforts. To fully make use of the high potential of IT in patient recruitment, many technical and process challenges have to be solved first. We contribute and discuss concrete technical challenges for patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration.

Architecture for a feasibility query portal for distributed COVID-19 Fast Healthcare Interoperability Resources (FHIR) patient data repositories: Design and Implementation Study (Preprint)

Background

An essential step in any medical research project after identifying the research question is to determine if there are sufficient patients available for a study and where to find them. Pursuing digital feasibility queries on available patient data registries has proven to be an excellent way of reusing existing real-world data sources. To support multicentric research, these feasibility queries should be designed and implemented to run across multiple sites and securely access local data. Working across hospitals usually involves working with different data formats and vocabularies. Recently, the Fast Healthcare Interoperability Resources (FHIR) standard was developed by Health Level Seven to address this concern and describe patient data in a standardized format. The Medical Informatics Initiative in Germany has committed to this standard and created data integration centers, which convert existing data into the FHIR format at each hospital. This partially solves the interoperability problem; however, a distributed feasibility query platform for the FHIR standard is still missing.

Objective

This study described the design and implementation of the components involved in creating a cross-hospital feasibility query platform for researchers based on FHIR resources. This effort was part of a large COVID-19 data exchange platform and was designed to be scalable for a broad range of patient data.

Methods

We analyzed and designed the abstract components necessary for a distributed feasibility query. This included a user interface for creating the query, backend with an ontology and terminology service, middleware for query distribution, and FHIR feasibility query execution service.

Results

We implemented the components described in the Methods section. The resulting solution was distributed to 33 German university hospitals. The functionality of the comprehensive network infrastructure was demonstrated using a test data set based on the German Corona Consensus Data Set. A performance test using specifically created synthetic data revealed the applicability of our solution to data sets containing millions of FHIR resources. The solution can be easily deployed across hospitals and supports feasibility queries, combining multiple inclusion and exclusion criteria using standard Health Level Seven query languages such as Clinical Quality Language and FHIR Search. Developing a platform based on multiple microservices allowed us to create an extendable platform and support multiple Health Level Seven query languages and middleware components to allow integration with future directions of the Medical Informatics Initiative.

Conclusions

We designed and implemented a feasibility platform for distributed feasibility queries, which works directly on FHIR-formatted data and distributed it across 33 university hospitals in Germany. We showed that developing a feasibility platform directly on the FHIR standard is feasible.

Generation of a Fast Healthcare Interoperability Resources (FHIR)-based Ontology for federated Feasibility Queries in the context of COVID-19: An automated approach (Preprint)

Background

The COVID-19 pandemic highlighted the importance of making research data from all German hospitals available to scientists to respond to current and future pandemics promptly. The heterogeneous data originating from proprietary systems at hospitals' sites must be harmonized and accessible. The German Corona Consensus Dataset (GECCO) specifies how data for COVID-19 patients will be standardized in Fast Healthcare Interoperability Resources (FHIR) profiles across German hospitals. However, given the complexity of the FHIR standard, the data harmonization is not sufficient to make the data accessible. A simplified visual representation is needed to reduce the technical burden, while allowing feasibility queries.

Objective

This study investigates how a search ontology can be automatically generated using FHIR profiles and a terminology server. Furthermore, it describes how this ontology can be used in a user interface (UI) and how a mapping and a terminology tree created together with the ontology can translate user input into FHIR queries.

Methods

We used the FHIR profiles from the GECCO data set combined with a terminology server to generate an ontology and the required mapping files for the translation. We analyzed the profiles and identified search criteria for the visual representation. In this process, we reduced the complex profiles to code value pairs for improved usability. We enriched our ontology with the necessary information to display it in a UI. We also developed an intermediate query language to transform the queries from the UI to federated FHIR requests. Separation of concerns resulted in discrepancies between the criteria used in the intermediate query format and the target query language. Therefore, a mapping was created to reintroduce all information relevant for creating the query in its target language. Further, we generated a tree representation of the ontology hierarchy, which allows resolving child concepts in the process.

Results

In the scope of this project, 82 (99%) of 83 elements defined in the GECCO profile were successfully implemented. We verified our solution based on an independently developed test patient. A discrepancy between the test data and the criteria was found in 6 cases due to different versions used to generate the test data and the UI profiles, the support for specific code systems, and the evaluation of postcoordinated Systematized Nomenclature of Medicine (SNOMED) codes. Our results highlight the need for governance mechanisms for version changes, concept mapping between values from different code systems encoding the same concept, and support for different unit dimensions.

Conclusions

We developed an automatic process to generate ontology and mapping files for FHIR-formatted data. Our tests found that this process works for most of our chosen FHIR profile criteria. The process established here works directly with FHIR profiles and a terminology server, making it extendable to other FHIR profiles and demonstrating that automatic ontology generation on FHIR profiles is feasible.

Challenges and Experiences Extending the cBioPortal for Cancer Genomics to a Molecular Tumor Board Platform

In Molecular Tumor Boards (MTBs), therapy recommendations for cancer patients are discussed. To aid decision-making based on the patient's molecular profile, the research platform cBioPortal was extended based on users' requirements. Additionally, a comprehensive dockerized workflow was developed to support the deployment of cBioPortal and connected services. In this work, we present the challenges and experiences of nearly two years of implementing and deploying an MTB platform based on cBioPortal and compare those to findings of a previous study.

[RECUR - Establishment of An Automated Digital Registry for Patients with Recurrent Stones in the Upper Urinary Tract]

Kidney stones, like cardiovascular diseases and diabetes mellitus, affect a large number of people. Patients suffer from acute pain, repeated hospitalizations and associated secondary diseases, such as arterial hypertension and renal insufficiency. This results in considerable costs for the society and its health care system. The recurrence rate is as high as 50%. The registry for RECurrent URolithiasis (RECUR) aims to fill existing evidence gaps. The prospective and longitudinal RECUR registry is funded by the German Ministry of Education and Science (BMBF). It is based on the digital infrastructure of the German Medical Informatics Initiative (MII). RECUR aims to include patients that have suffered from more than one stone occurrence and treated at any one of the ten participating university hospitals of the MIRACUM consortium. The intention is to obtain new information on risk factors and to evaluate different diagnosis and treatment algorithms. Along with the data form the patient's Electronic Health Records (EHR), the RECUR project will also collect Patient Reported Outcomes data from patients with recurrent kidney stones. These data will be collected at participating sites using digital questionnaires via a smartphone app. These data will be merged with medical data from the hospital information systems and saved in the MII research data repositories. The RECUR registry has a model character due to its fully federated, digital approach. This allows the recruitment of many patients, the collection of a wide range of data and their processing with low administrative and personnel costs.

Usability Evaluation of an Offline Electronic Data Capture App in a Prospective Multicenter Dementia Registry (digiDEM Bayern): Mixed Method Study

BACKGROUND

Digital registries have been shown to provide an efficient way of gaining a better understanding of the clinical complexity and long-term progression of diseases. The paperless method of electronic data capture (EDC) during a patient interview saves both time and resources. In the prospective multicenter project "Digital Dementia Registry Bavaria (digiDEM Bayern)," interviews are also performed on site in rural areas with unreliable internet connectivity. It must be ensured that EDC can still be performed in such a context and that there is no need to fall back on paper-based questionnaires. In addition to a web-based data collection solution, the EDC system REDCap (Research Electronic Data Capture) offers the option to collect data offline via an app and to synchronize it afterward.

OBJECTIVE

The aim of this study was to evaluate the usability of the REDCap app as an offline EDC option for a lay user group and to examine the necessary technology acceptance of using mobile devices for data collection. The feasibility of the app-based offline data collection in the digiDEM Bayern dementia registry project was then evaluated before going live.

METHODS

An exploratory mixed method design was employed in the form of an on-site usability test with the "Thinking Aloud" method combined with an online questionnaire including the System Usability Scale (SUS). The acceptance of mobile devices for data collection was surveyed based on five categories of the technology acceptance model.

RESULTS

Using the "Thinking Aloud" method, usability issues were identified and solutions were accordingly derived. Evaluation of the REDCap app resulted in a SUS score of 74, which represents "good" usability. After evaluating the technology acceptance questionnaire, it can be concluded that the lay user group is open to mobile devices as interview tools.

CONCLUSIONS

The usability evaluation results show that a lay user group generally agree that data collecting partners in the digiDEM project can handle the REDCap app well. The usability evaluation provided statements about positive aspects and could also identify usability issues relating to the REDCap app. In addition, the current technology acceptance in the sample showed that heterogeneous groups of different ages with diverse experiences in handling mobile devices are also ready for the use of app-based EDC systems. Based on these results, it can be assumed that the offline use of an app-based EDC system on mobile devices is a viable solution for collecting data in a decentralized registry-based research project.

Controlling my genome with my smartphone: first clinical experiences of the PROMISE system

Background

The development of Precision Medicine strategies requires high-dimensional phenotypic and genomic data, both of which are highly privacy-sensitive data types. Conventional data management systems lack the capabilities to sufficiently handle the expected large quantities of such sensitive data in a secure manner. PROMISE is a genetic data management concept that implements a highly secure platform for data exchange while preserving patient interests, privacy, and autonomy.

Methods

The concept of PROMISE to democratize genetic data was developed by an interdisciplinary team. It integrates a sophisticated cryptographic concept that allows only the patient to grant selective access to defined parts of his genetic information with single DNA base-pair resolution cryptography. The PROMISE system was developed for research purposes to evaluate the concept in a pilot study with nineteen cardiomyopathy patients undergoing genotyping, questionnaires, and longitudinal follow-up.

Results

The safety of genetic data was very important to 79%, and patients generally regarded the data as highly sensitive. More than half the patients reported that their attitude towards the handling of genetic data has changed after using the PROMISE app for 4 months (median). The patients reported higher confidence in data security and willingness to share their data with commercial third parties, including pharmaceutical companies (increase from 5 to 32%).

Conclusion

PROMISE democratizes genomic data by a transparent, secure, and patient-centric approach. This clinical pilot study evaluating a genetic data infrastructure is unique and shows that patient’s acceptance of data sharing can be increased by patient-centric decision-making.

Graphic abstract

Correction: Evaluation of Three Feasibility Tools for Identifying Patient Data and Biospecimen Availability: Comparative Usability Study

[This corrects the article DOI: 10.2196/25531.].

The journey to establishing an IT-infrastructure within the German Biobank Alliance

Background

Biobanks ensure the long-term storage and accessibility of biospecimens and corresponding data sets. Thus, they form the foundation for many research projects which may contribute to improving medical care. With the establishment of the German Biobank Node and Alliance, expertise in biobanking is bundled and strengthened. An important component within this research infrastructure is the set-up of an information technology (IT) network for allowing feasibility requests across individual biobanks.

Objective

We aim to describe relevant aspects that have shaped the journey to interconnect biobanks, to enhance their visibility within the research-community, to harmonize data, and to enable feasibility searches to support access to available data and biosamples.

Methods

To achieve this task, we resorted to a wide variety of methods: we ran a requirement analysis, decided on the mode of operation for the federated team of IT-developers and on the development approach itself, took related national and international initiatives into account, and concluded with evaluations of the developed software artefacts and the operation of the entire chain of applications.

Results

We drew an IT framework including all heterogeneous data aspects derived from our requirement analysis and developed a comprehensive IT infrastructure. The successful implementation benefited from a smooth interaction of a federated IT team distributed across all participating sites that was even able to manage a major technology change mid-project. Authentication and project management services from associated partners could be integrated and the graphic user interface for an intuitive search tool for biospecimens was designed iteratively. The developed code is open source to ensure sustainability and the local implementation is concluded and functioning. The evaluation of the components was positive.

Conclusions

The entire project had given ample opportunity for challenges, predictable and unpredictable—from the mode of operation to changing some of the initial ideas. We learned our lessons concerning personnel, budget planning and technical as well as manual monitoring as well as some requirements arising only during the process of the project. Nevertheless, we can here report a success story of a network infrastructure, highly agile and much easier in local installation than initially anticipated.

A comprehensive method protocol for annotation and integrated functional understanding of lncRNAs

Long non-coding RNAs (lncRNAs) are of fundamental biological importance; however, their functional role is often unclear or loosely defined as experimental characterization is challenging and bioinformatic methods are limited. We developed a novel integrated method protocol for the annotation and detailed functional characterization of lncRNAs within the genome. It combines annotation, normalization and gene expression with sequence-structure conservation, functional interactome and promoter analysis. Our protocol allows an analysis based on the tissue and biological context, and is powerful in functional characterization of experimental and clinical RNA-Seq datasets including existing lncRNAs. This is demonstrated on the uncharacterized lncRNA GATA6-AS1 in dilated cardiomyopathy.

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

Background  Many research initiatives aim at using data from electronic health records (EHRs) in observational studies. Participating sites of the German Medical Informatics Initiative (MII) established data integration centers to integrate EHR data within research data repositories to support local and federated analyses. To address concerns regarding possible data quality (DQ) issues of hospital routine data compared with data specifically collected for scientific purposes, we have previously presented a data quality assessment (DQA) tool providing a standardized approach to assess DQ of the research data repositories at the MIRACUM consortium's partner sites.

Objectives  Major limitations of the former approach included manual interpretation of the results and hard coding of analyses, making their expansion to new data elements and databases time-consuming and error prone. We here present an enhanced version of the DQA tool by linking it to common data element definitions stored in a metadata repository (MDR), adopting the harmonized DQA framework from Kahn et al and its application within the MIRACUM consortium.

Methods  Data quality checks were consequently aligned to a harmonized DQA terminology. Database-specific information were systematically identified and represented in an MDR. Furthermore, a structured representation of logical relations between data elements was developed to model plausibility-statements in the MDR.

Results  The MIRACUM DQA tool was linked to data element definitions stored in a consortium-wide MDR. Additional databases used within MIRACUM were linked to the DQ checks by extending the respective data elements in the MDR with the required information. The evaluation of DQ checks was automated. An adaptable software implementation is provided with the R package DQAstats .

Conclusion  The enhancements of the DQA tool facilitate the future integration of new data elements and make the tool scalable to other databases and data models. It has been provided to all ten MIRACUM partners and was successfully deployed and integrated into their respective data integration center infrastructure.

refineR: A Novel Algorithm for Reference Interval Estimation from Real-World Data

Reference intervals are essential for the interpretation of laboratory test results in medicine. We propose a novel indirect approach to estimate reference intervals from real-world data as an alternative to direct methods, which require samples from healthy individuals. The presented refineR algorithm separates the non-pathological distribution from the pathological distribution of observed test results using an inverse approach and identifies the model that best explains the non-pathological distribution. To evaluate its performance, we simulated test results from six common laboratory analytes with a varying location and fraction of pathological test results. Estimated reference intervals were compared to the ground truth, an alternative indirect method (kosmic), and the direct method (N = 120 and N = 400 samples). Overall, refineR achieved the lowest mean percentage error of all methods (2.77%). Analyzing the amount of reference intervals within ± 1 total error deviation from the ground truth, refineR (82.5%) was inferior to the direct method with N = 400 samples (90.1%), but outperformed kosmic (70.8%) and the direct method with N = 120 (67.4%). Additionally, reference intervals estimated from pediatric data were comparable to published direct method studies. In conclusion, the refineR algorithm enables precise estimation of reference intervals from real-world data and represents a viable complement to the direct method.

Time Tracking of Standard Ultrasound Examinations in Pediatric Hospitals and Pediatric Medical Practices - A Multicenter Study by the Pediatric Section of the German Society of Ultrasound in Medicine (DEGUM)

PURPOSE

 Ultrasonography is the primary imaging modality in pediatrics but still lacks sufficient reimbursement in Germany. In this multicenter study, national data for the duration of standard ultrasound in pediatrics were systematically documented in order to specify the actual time required.

MATERIALS AND METHODS

 N = 10 hospitals (N = 5 university hospitals, N = 5 non-university hospitals) and N = 3 medical practices in Germany recorded the entire process of an ultrasound examination in a special protocol developed by the Pediatric Section of the DEGUM. The duration of each of seven single steps during ultrasonography (from data input to final discussion of the results) of different organ systems was logged.

RESULTS

 In total, N = 2118 examinations from different organ systems were recorded. N = 10 organ systems were examined frequently (> 30 times). The total duration of an ultrasound examination was statistically significantly longer in hospitals compared to medical practices (median (IQR) 27 min. (18-38) vs. 12 min. (9-17), p < 0.001). The "hands-on" patient time was approximately one half of the total required time in both settings (49.9 % vs. 48.9 %). Ultrasonography of the abdomen and brain lasted longer in university hospitals than in non-university hospitals (p < 0.001, and p = 0.04, respectively). Cooperation and age did not uniformly correlate with the total duration.

CONCLUSION

 This study provides novel comprehensive national data for the duration of standardized ultrasound examinations of children and adolescents in Germany. These data are essential for a further evaluation of the economic costs and should support better remuneration in the future.

Evaluation of Three Feasibility Tools for Identifying Patient Data and Biospecimen Availability: Comparative Usability Study

BACKGROUND

To meet the growing importance of real-word data analysis, clinical data and biosamples must be timely made available. Feasibility platforms are often the first contact point for determining the availability of such data for specific research questions. Therefore, a user-friendly interface should be provided to enable access to this information easily. The German Medical Informatics Initiative also aims to establish such a platform for its infrastructure. Although some of these platforms are actively used, their tools still have limitations. Consequently, the Medical Informatics Initiative consortium MIRACUM (Medical Informatics in Research and Care in University Medicine) committed itself to analyzing the pros and cons of existing solutions and to designing an optimized graphical feasibility user interface.

OBJECTIVE

The aim of this study is to identify the system that is most user-friendly and thus forms the best basis for developing a harmonized tool. To achieve this goal, we carried out a comparative usability evaluation of existing tools used by researchers acting as end users.

METHODS

The evaluation included three preselected search tools and was conducted as a qualitative exploratory study with a randomized design over a period of 6 weeks. The tools in question were the MIRACUM i2b2 (Informatics for Integrating Biology and the Bedside) feasibility platform, OHDSI's (Observational Health Data Sciences and Informatics) ATLAS, and the Sample Locator of the German Biobank Alliance. The evaluation was conducted in the form of a web-based usability test (usability walkthrough combined with a web-based questionnaire) with participants aged between 26 and 63 years who work as medical doctors.

RESULTS

In total, 17 study participants evaluated the three tools. The overall evaluation of usability, which was based on the System Usability Scale, showed that the Sample Locator, with a mean System Usability Scale score of 77.03 (SD 20.62), was significantly superior to the other two tools (Wilcoxon test; Sample Locator vs i2b2: P=.047; Sample Locator vs ATLAS: P=.001). i2b2, with a score of 59.83 (SD 25.36), performed significantly better than ATLAS, which had a score of 27.81 (SD 21.79; Wilcoxon test; i2b2 vs ATLAS: P=.005). The analysis of the material generated by the usability walkthrough method confirmed these findings. ATLAS caused the most usability problems (n=66), followed by i2b2 (n=48) and the Sample Locator (n=22). Moreover, the Sample Locator achieved the highest ratings with respect to additional questions regarding satisfaction with the tools.

CONCLUSIONS

This study provides data to develop a suitable basis for the selection of a harmonized tool for feasibility studies via concrete evaluation and a comparison of the usability of three different types of query builders. The feedback obtained from the participants during the usability test made it possible to identify user problems and positive design aspects of the individual tools and compare them qualitatively.

Designing and Implementing an IT Architecture for a Digital Multicenter Dementia Registry: digiDEM Bayern

Background  Registries are an essential research tool to investigate the long-term course of diseases and their impact on the affected. The project digiDEM Bayern will set up a prospective dementia registry to collect long-term data of people with dementia and their caregivers in Bavaria (Germany) supported by more than 300 research partners.

Objective  The objective of this article is to outline an information technology (IT) architecture for the integration of a registry and comprehensive participant management in a dementia study. Measures to ensure high data quality, study governance, along with data privacy, and security are to be included in the architecture.

Methods  The architecture was developed based on an iterative, stakeholder-oriented process. The development was inspired by the Twin Peaks Model that focuses on the codevelopment of requirements and architecture. We gradually moved from a general to a detailed understanding of both the requirements and design through a series of iterations. The experience learned from the pilot phase was integrated into a further iterative process of continuous improvement of the architecture.

Results  The infrastructure provides a standardized workflow to support the electronic data collection and trace each participant's study process. Therefore, the implementation consists of three systems: (1) electronic data capture system for Web-based or offline app-based data collection; (2) participant management system for the administration of the identity data of participants and research partners as well as of the overall study governance process; and (3) videoconferencing software for conducting interviews online. First experiences in the pilot phase have proven the feasibility of the framework.

Conclusion  This article outlines an IT architecture to integrate a registry and participant management in a dementia research project. The framework was discussed and developed with the involvement of numerous stakeholders. Due to its adaptability of used software systems, a transfer to other projects should be easily possible.

BiasCorrector: Fast and accurate correction of all types of experimental biases in quantitative DNA methylation data derived by different technologies

Quantification of DNA methylation in neoplastic cells is crucial both from mechanistic and diagnostic perspectives. However, such measurements are prone to different experimental biases. Polymerase chain reaction (PCR) bias results in an unequal recovery of methylated and unmethylated alleles at the sample preparation step. Post-PCR biases get introduced additionally by the readout processes. Correcting the biases is more practicable than optimising experimental conditions, as demonstrated previously. However, utilisation of our earlier developed algorithm strongly necessitates automation. Here, we present two R packages: rBiasCorrection, the core algorithms to correct biases; and BiasCorrector, its web-based graphical user interface frontend. The software detects and analyses experimental biases in calibration DNA samples at a single base resolution by using cubic polynomial and hyperbolic regression. The correction coefficients from the best regression type are employed to compensate for the bias. Three common technologies-bisulphite pyrosequencing, next-generation sequencing and oligonucleotide microarrays-were used to comprehensively test BiasCorrector. We demonstrate the accuracy of BiasCorrector's performance and reveal technology-specific PCR- and post-PCR biases. BiasCorrector effectively eliminates biases regardless of their nature, locus, the number of interrogated methylation sites and the detection method, thus representing a user-friendly tool for producing accurate epigenetic results.

A Multi-User Terminology Mapping Toolbox

Semantic interoperability is a major challenge in multi-center data sharing projects, a challenge that the German Initiative for Medical Informatics is taking up. With respect to laboratory data, enriching site-specific tests and measurements with LOINC codes appears to be a crucial step in supporting cross-institutional research. However, this effort is very time-consuming, as it requires expert knowledge of local site specifics. To ease this process, we developed a generic manual collaborative terminology mapping tool, the MIRACUM Mapper. It allows the creation of arbitrary mapping workflows involving different user roles. A mapping workflow with two user roles has been implemented at University Hospital Erlangen to support the local LOINC mapping. Additionally, the MIRACUM LabVisualizeR provides summary statistics and visualizations of analyte data. We developed a toolbox that facilitates the collaborative creation of mappings and streamlines the review as well as the validation process. The two tools are available under an open source license.

Visualization of Similar Patients in a Clinical Decision Support System for Rare Diseases - A Focus Group Study

The diagnosis of patients with rare diseases is often delayed. A Clinical Decision Support System using similarity analysis of patient-based data may have the potential to support the diagnosis of patients with rare diseases. This qualitative study has the objective to investigate how the result of a patient similarity analysis should be presented to a physician to enable diagnosis support. We conducted a focus group with physicians practicing in rare diseases as well as medical informatics researchers. To prepare the focus group, a literature search was performed to check the current state of research regarding visualization of similar patients. We then created software-mockups for the presentation of these visualization methods for the discussion within the focus group. Two persons took independently field notes for data collection of the focus group. A questionnaire was distributed to the participants to rate the visualization methods. The results show that four visualization methods are promising for the visualization of similar patients: "Patient on demand table", "Criteria selection", "Time-Series chart" and "Patient timeline. "Patient on demand table" shows a direct comparison of patient characteristics, whereas "Criteria selection" allows the selection of different patient criteria to get deeper insights into the data. The "Time-Series chart" shows the time course of clinical parameters (e.g. blood pressure) whereas a "Patient timeline" indicates which time events exist for a patient (e.g. several symptoms on different dates). In the future, we will develop a software-prototype of the Clinical Decision Support System to include the visualization methods and evaluate the clinical usage.

First Steps to Evaluate an NLP Tool's Medication Extraction Accuracy from Discharge Letters

INTRODUCTION

The aim of this study is to evaluate the use of a natural language processing (NLP) software to extract medication statements from unstructured medical discharge letters.

METHODS

Ten randomly selected discharge letters were extracted from the data warehouse of the University Hospital Erlangen (UHE) and manually annotated to create a gold standard. The AHD NLP tool, provided by MIRACUM's industry partner was used to annotate these discharge letters. Annotations by the NLP tool where then compared to the gold standard on two levels: phrase precision (whether or not the whole medication statement has been identified correctly) and token precision (whether or not the medication name has been identified correctly within correctly discovered medication phrases).

RESULTS

The NLP tool detected medication related phrases with an overall F-measure of 0.852. The medication name has been identified correctly with an overall F-measure of 0.936.

DISCUSSION

This proof-of-concept study is a first step towards an automated scalable evaluation system for MIRACUM's industry partner's NLP tool by using a gold standard. Medication phrases and names have been correctly identified in most cases by the NLP system. Future effort needs to be put into extending and validating the gold standard.

eHealth Interventions for Dementia - Using WordPress Plugins as a Flexible Dissemination for Dementia Service Providers

The benefits of eHealth interventions for people with dementia and their informal caregivers have been demonstrated in several studies. In times of contact restrictions, digital solutions have become increasingly important, especially for people with dementia and their mostly elderly caregiving relatives, which are at increased risk of severe illness from COVID-19. As in many other health areas, there is a lack of digital interventions in the dementia landscape that are successfully implemented (i.e., put into practice), especially digital interventions that are scientifically evaluated. Evaluated and proven effective digital interventions exist, but these often do not find their way from research into practice and stay on low-level implementation readiness. Within the project digiDEM Bayern, a digital platform with digital services and interventions for people affected by dementia (people with dementia, caregivers, volunteers and interested citizens) is established. As one digital intervention for informal caregivers, the 'Angehörigenampel' (caregivers' traffic-light) was developed, which is able to assess the physical and psychological burden of caregivers. This can help to counteract the health effects of caregiving burden early on before it is too late. The development of the digital intervention as a WordPress-plugin was kept generic so that it can easily be adapted to other languages on further websites. The 'intervention as a plugin' approach demonstrates an easy and flexible way of deploying eHealth interventions to other service providers, especially from other countries. The implementation barriers for other service providers are low enough for them to be able to easily integrate the eHealth intervention on their website, enabling more caregivers to benefit from the disseminated eHealth intervention.

A Framework for Criteria-Based Selection and Processing of Fast Healthcare Interoperability Resources (FHIR) Data for Statistical Analysis: Design and Implementation Study

Background

The harmonization and standardization of digital medical information for research purposes is a challenging and ongoing collaborative effort. Current research data repositories typically require extensive efforts in harmonizing and transforming original clinical data. The Fast Healthcare Interoperability Resources (FHIR) format was designed primarily to represent clinical processes; therefore, it closely resembles the clinical data model and is more widely available across modern electronic health records. However, no common standardized data format is directly suitable for statistical analyses, and data need to be preprocessed before statistical analysis.

Objective

This study aimed to elucidate how FHIR data can be queried directly with a preprocessing service and be used for statistical analyses.

Methods

We propose that the binary JavaScript Object Notation format of the PostgreSQL (PSQL) open source database is suitable for not only storing FHIR data, but also extending it with preprocessing and filtering services, which directly transform data stored in FHIR format into prepared data subsets for statistical analysis. We specified an interface for this preprocessor, implemented and deployed it at University Hospital Erlangen-Nürnberg, generated 3 sample data sets, and analyzed the available data.

Results

We imported real-world patient data from 2016 to 2018 into a standard PSQL database, generating a dataset of approximately 35.5 million FHIR resources, including “Patient,” “Encounter,” “Condition” (diagnoses specified using International Classification of Diseases codes), “Procedure,” and “Observation” (laboratory test results). We then integrated the developed preprocessing service with the PSQL database and the locally installed web-based KETOS analysis platform. Advanced statistical analyses were feasible using the developed framework using 3 clinically relevant scenarios (data-driven establishment of hemoglobin reference intervals, assessment of anemia prevalence in patients with cancer, and investigation of the adverse effects of drugs).

Conclusions

This study shows how the standard open source database PSQL can be used to store FHIR data and be integrated with a specifically developed preprocessing and analysis framework. This enables dataset generation with advanced medical criteria and the integration of subsequent statistical analysis. The web-based preprocessing service can be deployed locally at the hospital level, protecting patients’ privacy while being integrated with existing open source data analysis tools currently being developed across Germany.

Evaluation of a clinical decision support system for rare diseases: a qualitative study

BACKGROUND

Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS.

METHODS

We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS).

RESULTS

A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability.

CONCLUSIONS

This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.

High-resolution pediatric reference intervals for 15 biochemical analytes described using fractional polynomials

OBJECTIVES

Assessment of children's laboratory test results requires consideration of the extensive changes that occur during physiological development and result in pronounced sex- and age-specific dynamics in many biochemical analytes. Pediatric reference intervals have to account for these dynamics, but ethical and practical challenges limit the availability of appropriate pediatric reference intervals that cover children from birth to adulthood. We have therefore initiated the multi-center data-driven PEDREF project (Next-Generation Pediatric Reference Intervals) to create pediatric reference intervals using data from laboratory information systems.

METHODS

We analyzed laboratory test results from 638,683 patients (217,883-982,548 samples per analyte, a median of 603,745 test results per analyte, and 10,298,067 test results in total) performed during patient care in 13 German centers. Test results from children with repeat measurements were discarded, and we estimated the distribution of physiological test results using a validated statistical approach (kosmic).

RESULTS

We report continuous pediatric reference intervals and percentile charts for alanine transaminase, aspartate transaminase, lactate dehydrogenase, alkaline phosphatase, γ-glutamyl-transferase, total protein, albumin, creatinine, urea, sodium, potassium, calcium, chloride, anorganic phosphate, and magnesium. Reference intervals are provided as tables and fractional polynomial functions (i.e., mathematical equations) that can be integrated into laboratory information systems. Additionally, Z-scores and percentiles enable the normalization of test results by age and sex to facilitate their interpretation across age groups.

CONCLUSIONS

The provided reference intervals and percentile charts enable precise assessment of laboratory test results in children from birth to adulthood. Our findings highlight the pronounced dynamics in many biochemical analytes in neonates, which require particular consideration in reference intervals to support clinical decision making most effectively.

Digital Dementia Registry Bavaria-digiDEM Bayern: study protocol for a multicentre, prospective, longitudinal register study

INTRODUCTION

Dementia is one of the most relevant widespread diseases, with a prevalence of currently 50 million people with dementia worldwide. The care of people with dementia will be one of the major challenges for healthcare systems worldwide. Digitalisation offers new possibilities to improve both dementia healthcare and health outcomes research as a fundament for national healthcare planning. The 'Digital Dementia Registry Bavaria-digiDEM Bayern' aims to improve the understanding of the complexity and long-term progression of dementia and the current care situation in Bavaria. Moreover, by offering digital services, digiDEM will actively contribute to improving the care situation in Bavaria. METHODS AND ANALYSIS: digiDEM will recruit people with dementia and their family caregivers in all administrative regions of Bavaria. All participants will undergo dementia screening prior to study inclusion in order to identify people with mild cognitive impairment and mild-to-moderate dementia. Participants will be followed up over a period of three years. Sociodemographic data, type of dementia, symptoms, diagnosis, cognitive trajectories, activities of daily living, behavioural and psychological symptoms, falls, resource utilisation, caregiver burden, quality of life, needs of people with dementia and their caregivers, mobility, use of media and sources of information will be assessed. The project will implement a digital web-based platform for data collection. Data will be collected by means of standardised online or face-to-face interviews.

ETHICS AND DISSEMINATION

The study obtained ethical approval from the Ethics Committee of the Medical Faculty of Friedrich-Alexander-University Erlangen-Nürnberg (FAU) (application number: 253_20 B). Findings will be used for evidence-based decision-making for health decision-makers in order to optimise dementia healthcare in the state of Bavaria. Specific analyses will be conducted for the participating research partners. Results of the study will be published in peer-reviewed journals.

Patient Cohort Identification on Time Series Data Using the OMOP Common Data Model

BACKGROUND

The identification of patient cohorts for recruiting patients into clinical trials requires an evaluation of study-specific inclusion and exclusion criteria. These criteria are specified depending on corresponding clinical facts. Some of these facts may not be present in the clinical source systems and need to be calculated either in advance or at cohort query runtime (so-called feasibility query).

OBJECTIVES

We use the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) as the repository for our clinical data. However, Atlas, the graphical user interface of OMOP, does not offer the functionality to perform calculations on facts data. Therefore, we were in search for a different approach. The objective of this study is to investigate whether the Arden Syntax can be used for feasibility queries on the OMOP CDM to enable on-the-fly calculations at query runtime, to eliminate the need to precalculate data elements that are involved with researchers' criteria specification.

METHODS

We implemented a service that reads the facts from the OMOP repository and provides it in a form which an Arden Syntax Medical Logic Module (MLM) can process. Then, we implemented an MLM that applies the eligibility criteria to every patient data set and outputs the list of eligible cases (i.e., performs the feasibility query).

RESULTS

The study resulted in an MLM-based feasibility query that identifies cases of overventilation as an example of how an on-the-fly calculation can be realized. The algorithm is split into two MLMs to provide the reusability of the approach.

CONCLUSION

We found that MLMs are a suitable technology for feasibility queries on the OMOP CDM. Our method of performing on-the-fly calculations can be employed with any OMOP instance and without touching existing infrastructure like the Extract, Transform and Load pipeline. Therefore, we think that it is a well-suited method to perform on-the-fly calculations on OMOP.