Secondary use of routinely collected patient data in a clinical trial: an evaluation of the effects on patient recruitment and data acquisition

Effectiveness of IT-supported patient recruitment: study protocol for an interrupted time series study at ten German university hospitals

BACKGROUND

As part of the German Medical Informatics Initiative, the MIRACUM project establishes data integration centers across ten German university hospitals. The embedded MIRACUM Use Case "Alerting in Care - IT Support for Patient Recruitment", aims to support the recruitment into clinical trials by automatically querying the repositories for patients satisfying eligibility criteria and presenting them as screening candidates. The objective of this study is to investigate whether the developed recruitment tool has a positive effect on study recruitment within a multi-center environment by increasing the number of participants. Its secondary objective is the measurement of organizational burden and user satisfaction of the provided IT solution.

METHODS

The study uses an Interrupted Time Series Design with a duration of 15 months. All trials start in the control phase of randomized length with regular recruitment and change to the intervention phase with additional IT support. The intervention consists of the application of a recruitment-support system which uses patient data collected in general care for screening according to specific criteria. The inclusion and exclusion criteria of all selected trials are translated into a machine-readable format using the OHDSI ATLAS tool. All patient data from the data integration centers is regularly checked against these criteria. The primary outcome is the number of participants recruited per trial and week standardized by the targeted number of participants per week and the expected recruitment duration of the specific trial. Secondary outcomes are usability, usefulness, and efficacy of the recruitment support. Sample size calculation based on simple parallel group assumption can demonstrate an effect size of d=0.57 on a significance level of 5% and a power of 80% with a total number of 100 trials (10 per site). Data describing the included trials and the recruitment process is collected at each site. The primary analysis will be conducted using linear mixed models with the actual recruitment number per week and trial standardized by the expected recruitment number per week and trial as the dependent variable.

DISCUSSION

The application of an IT-supported recruitment solution developed in the MIRACUM consortium leads to an increased number of recruited participants in studies at German university hospitals. It supports employees engaged in the recruitment of trial participants and is easy to integrate in their daily work.

A Centralized EHR-Based Model for the Recruitment of Rural and Lower Socioeconomic Participants in Pragmatic Trials: A Secondary Analysis of the Diuretic Comparison Project

Importance

Participant diversity is important for reducing study bias and increasing generalizability of comparative effectiveness research.

Objective

Demonstrate the operational efficiency of a centralized electronic health record (EHR)-based model for recruiting difficult-to-reach participants in a pragmatic trial.

Design, Setting, and Participants

This comparative effectiveness study was a secondary analysis of Diuretic Comparison Project, a randomized clinical trial conducted between 2016 and 2022 (mean [SD] follow-up, 2.4 [1.4] years) comparing 2 commonly prescribed antihypertensives, which used an EHR-based recruitment model. Electronic study workflows, in tandem with routine clinical practice, were adapted by 72 Veteran Affairs (VA) primary care networks. Data were analyzed from August to December 2022.

Main Outcomes and Measures

Measures reflecting recruitment capacity (monthly rate), operational efficiency (median time for completion of electronic procedures), and geographic reach (percentage of patients recruited from rural areas) were examined.

Results

A total of 13 523 patients with hypertension (mean [SD] age, 72 [5.4] years; 13 092 male [96.8%]) were recruited from 537 outpatient clinics. Approximately 205 patients were randomized per month and a median of 35 days (Q1-Q3, 23-80 days) was needed to complete electronic recruitment. The annual income was below the national median for 69% of the cohort. Patients from all 50 states, Puerto Rico, and the District of Columbia were included and 45% resided in rural areas.

Conclusions and Relevance

In this secondary analysis of a multicenter pragmatic trial, a centralized EHR-based recruitment model was associated with improved participation from underrepresented groups. These participants often are difficult to reach, with their exclusion potentially biasing trial results; eliminating in-person study visits and local site involvement can minimize barriers for the recruitment of patients from rural and lower socioeconomic areas.

Trial Registration

The Diuretic Comparison Project (DCP) was registered on ClinicalTrials.gov Identifier: NCT02185417.

How to Improve Cancer Patients ENrollment in Clinical Trials From rEal-Life Databases Using the Observational Medical Outcomes Partnership Oncology Extension: Results of the PENELOPE Initiative in Urologic Cancers

PURPOSE

To compare the computability of Observational Medical Outcomes Partnership (OMOP)-based queries related to prescreening of patients using two versions of the OMOP common data model (CDM; v5.3 and v5.4) and to assess the performance of the Greater Paris University Hospital (APHP) prescreening tool.

MATERIALS AND METHODS

We identified the prescreening information items being relevant for prescreening of patients with cancer. We randomly selected 15 academic and industry-sponsored urology phase I-IV clinical trials (CTs) launched at APHP between 2016 and 2021. The computability of the related prescreening criteria (PC) was defined by their translation rate in OMOP-compliant queries and by their execution rate on the APHP clinical data warehouse (CDW) containing data of 205,977 patients with cancer. The overall performance of the prescreening tool was assessed by the rate of true- and false-positive cases of three randomly selected CTs.

RESULTS

We defined a list of 15 minimal information items being relevant for patients' prescreening. We identified 83 PC of the 534 eligibility criteria from the 15 CTs. We translated 33 and 62 PC in queries on the basis of OMOP CDM v5.3 and v5.4, respectively (translation rates of 40% and 75%, respectively). Of the 33 PC translated in the v5.3 of the OMOP CDM, 19 could be executed on the APHP CDW (execution rate of 58%). Of 83 PC, the computability rate on the APHP CDW reached 23%. On the basis of three CTs, we identified 17, 32, and 63 patients as being potentially eligible for inclusion in those CTs, resulting in positive predictive values of 53%, 41%, and 21%, respectively.

CONCLUSION

We showed that PC could be formalized according to the OMOP CDM and that the oncology extension increased their translation rate through better representation of cancer natural history.

Applicability of Different Electronic Record Types for Use in Patient Recruitment Support Systems: Comparative Analysis

Background

Clinical trials constitute an important pillar in medical research. It is beneficial to support recruitment for clinical trials using software tools, so-called patient recruitment support systems; however, such information technology systems have not been frequently used to date. Because medical information systems' underlying data collection methods strongly influence the benefits of implementing patient recruitment support systems, we investigated patient recruitment support system requirements and corresponding electronic record types such as electronic medical record, electronic health record, electronic medical case record, personal health record, and personal cross-enterprise health record.

Objective

The aim of this study was to (1) define requirements for successful patient recruitment support system deployment and (2) differentiate and compare patient recruitment support system–relevant properties of different electronic record types.

Methods

In a previous study, we gathered requirements for patient recruitment support systems from literature and unstructured interviews with stakeholders (15 patients, 3 physicians, 5 data privacy experts, 4 researchers, and 5 staff members of hospital administration). For this investigation, the requirements were amended and categorized based on input from scientific sessions. Based on literature with a focus on patient recruitment support system–relevant properties, different electronic record types (electronic medical record, electronic health record, electronic medical case record, personal health record and personal cross-enterprise health record) were described in detail. We also evaluated which patient recruitment support system requirements can be achieved for each electronic record type.

Results

Patient recruitment support system requirements (n=16) were grouped into 4 categories (consent management, patient recruitment management, trial management, and general requirements). All 16 requirements could be partially met by at least 1 type of electronic record. Only 1 requirement was fully met by all 5 types. According to our analysis, personal cross-enterprise health records fulfill most requirements for patient recruitment support systems. They demonstrate advantages especially in 2 domains (1) supporting patient empowerment and (2) granting access to the complete medical history of patients.

Conclusions

In combination with patient recruitment support systems, personal cross-enterprise health records prove superior to other electronic record types, and therefore, this integration approach should be further investigated.

Secondary Use of Clinical Data in Data-Gathering, Non-Interventional Research or Learning Activities: Definition, Types, and a Framework for Risk Assessment

Background

The secondary use of clinical data in data-gathering, non-interventional research or learning activities (SeConts) has great potential for scientific progress and health care improvement. At the same time, it poses relevant risks for the privacy and informational self-determination of patients whose data are used.

Objective

Since the current literature lacks a tailored framework for risk assessment in SeConts as well as a clarification of the concept and practical scope of SeConts, we aim to fill this gap.

Methods

In this study, we analyze each element of the concept of SeConts to provide a synthetic definition, investigate the practical relevance and scope of SeConts through a literature review, and operationalize the widespread definition of risk (as a harmful event of a certain magnitude that occurs with a certain probability) to conduct a tailored analysis of privacy risk factors typically implied in SeConts.

Results

We offer a conceptual clarification and definition of SeConts and provide a list of types of research and learning activities that can be subsumed under the definition of SeConts. We also offer a proposal for the classification of SeConts types into the categories non-interventional (observational) clinical research, quality control and improvement, or public health research. In addition, we provide a list of risk factors that determine the probability or magnitude of harm implied in SeConts. The risk factors provide a framework for assessing the privacy-related risks for patients implied in SeConts. We illustrate the use of risk assessment by applying it to a concrete example.

Conclusions

In the future, research ethics committees and data use and access committees will be able to rely on and apply the framework offered here when reviewing projects of secondary use of clinical data for learning and research purposes.

Leveraging Real-World Data for the Selection of Relevant Eligibility Criteria for the Implementation of Electronic Recruitment Support in Clinical Trials

BACKGROUND

Even though clinical trials are indispensable for medical research, they are frequently impaired by delayed or incomplete patient recruitment, resulting in cost overruns or aborted studies. Study protocols based on real-world data with precisely expressed eligibility criteria and realistic cohort estimations are crucial for successful study execution. The increasing availability of routine clinical data in electronic health records (EHRs) provides the opportunity to also support patient recruitment during the prescreening phase. While solutions for electronic recruitment support have been published, to our knowledge, no method for the prioritization of eligibility criteria in this context has been explored.

METHODS

In the context of the Electronic Health Records for Clinical Research (EHR4CR) project, we examined the eligibility criteria of the KATHERINE trial. Criteria were extracted from the study protocol, deduplicated, and decomposed. A paper chart review and data warehouse query were executed to retrieve clinical data for the resulting set of simplified criteria separately from both sources. Criteria were scored according to disease specificity, data availability, and discriminatory power based on their content and the clinical dataset.

RESULTS

The study protocol contained 35 eligibility criteria, which after simplification yielded 70 atomic criteria. For a cohort of 106 patients with breast cancer and neoadjuvant treatment, 47.9% of data elements were captured through paper chart review, with the data warehouse query yielding 26.9% of data elements. Score application resulted in a prioritized subset of 17 criteria, which yielded a sensitivity of 1.00 and specificity 0.57 on EHR data (paper charts, 1.00 and 0.80) compared with actual recruitment in the trial.

CONCLUSION

It is possible to prioritize clinical trial eligibility criteria based on real-world data to optimize prescreening of patients on a selected subset of relevant and available criteria and reduce implementation efforts for recruitment support. The performance could be further improved by increasing EHR data coverage.

Linkage of the CHHiP randomised controlled trial with primary care data: a study investigating ways of supplementing cancer trials and improving evidence-based practice

BACKGROUND

Randomised controlled trials (RCTs) are the gold standard for evidence-based practice. However, RCTs can have limitations. For example, translation of findings into practice can be limited by design features, such as inclusion criteria, not accurately reflecting clinical populations. In addition, it is expensive to recruit and follow-up participants in RCTs. Linkage with routinely collected data could offer a cost-effective way to enhance the conduct and generalisability of RCTs. The aim of this study is to investigate how primary care data can support RCTs.

METHODS

Secondary analysis following linkage of two datasets: 1) multicentre CHHiP radiotherapy trial (ISRCTN97182923) and 2) primary care database from the Royal College of General Practitioners Research and Surveillance Centre. Comorbidities and medications recorded in CHHiP at baseline, and radiotherapy-related toxicity recorded in CHHiP over time were compared with primary care records. The association of comorbidities and medications with toxicity was analysed with mixed-effects logistic regression.

RESULTS

Primary care records were extracted for 106 out of 2811 CHHiP participants recruited from sites in England (median age 70, range 44 to 82). Complementary information included longitudinal body mass index, blood pressure and cholesterol, as well as baseline smoking and alcohol usage but was limited by the considerable missing data. In the linked sample, 9 (8%) participants were recorded in CHHiP as having a history of diabetes and 38 (36%) hypertension, whereas primary care records indicated incidence prior to trial entry of 11 (10%) and 40 (38%) respectively. Concomitant medications were not collected in CHHiP but available in primary care records. This indicated that 44 (41.5%) men took aspirin, 65 (61.3%) statins, 14 (13.2%) metformin and 46 (43.4%) phosphodiesterase-5-inhibitors at some point before or after trial entry.

CONCLUSIONS

We provide a set of recommendations on linkage and supplementation of trials. Data recorded in primary care are a rich resource and linkage could provide near real-time information to supplement trials and an efficient and cost-effective mechanism for long-term follow-up. In addition, standardised primary care data extracts could form part of RCT recruitment and conduct. However, this is at present limited by the variable quality and fragmentation of primary care data.

Digital tools for the recruitment and retention of participants in randomised controlled trials: a systematic map

BACKGROUND

Recruiting and retaining participants in randomised controlled trials (RCTs) is challenging. Digital tools, such as social media, data mining, email or text-messaging, could improve recruitment or retention, but an overview of this research area is lacking. We aimed to systematically map the characteristics of digital recruitment and retention tools for RCTs, and the features of the comparative studies that have evaluated the effectiveness of these tools during the past 10 years.

METHODS

We searched Medline, Embase, other databases, the Internet, and relevant web sites in July 2018 to identify comparative studies of digital tools for recruiting and/or retaining participants in health RCTs. Two reviewers independently screened references against protocol-specified eligibility criteria. Included studies were coded by one reviewer with 20% checked by a second reviewer, using pre-defined keywords to describe characteristics of the studies, populations and digital tools evaluated.

RESULTS

We identified 9163 potentially relevant references, of which 104 articles reporting 105 comparative studies were included in the systematic map. The number of published studies on digital tools has doubled in the past decade, but most studies evaluated digital tools for recruitment rather than retention. The key health areas investigated were health promotion, cancers, circulatory system diseases and mental health. Few studies focussed on minority or under-served populations, and most studies were observational. The most frequently-studied digital tools were social media, Internet sites, email and tv/radio for recruitment; and email and text-messaging for retention. One quarter of the studies measured efficiency (cost per recruited or retained participant) but few studies have evaluated people's attitudes towards the use of digital tools.

CONCLUSIONS

This systematic map highlights a number of evidence gaps and may help stakeholders to identify and prioritise further research needs. In particular, there is a need for rigorous research on the efficiency of the digital tools and their impact on RCT participants and investigators, perhaps as studies-within-a-trial (SWAT) research. There is also a need for research into how digital tools may improve participant retention in RCTs which is currently underrepresented relative to recruitment research.

REGISTRATION

Not registered; based on a pre-specified protocol, peer-reviewed by the project's Advisory Board.

Using digital tools in the recruitment and retention in randomised controlled trials: survey of UK Clinical Trial Units and a qualitative study

BACKGROUND

Recruitment and retention of participants in randomised controlled trials (RCTs) is a key determinant of success but is challenging. Trialists and UK Clinical Research Collaboration (UKCRC) Clinical Trials Units (CTUs) are increasingly exploring the use of digital tools to identify, recruit and retain participants. The aim of this UK National Institute for Health Research (NIHR) study was to identify what digital tools are currently used by CTUs and understand the performance characteristics required to be judged useful.

METHODS

A scoping of searches (and a survey with NIHR funding staff), a survey with all 52 UKCRC CTUs and 16 qualitative interviews were conducted with five stakeholder groups including trialists within CTUs, funders and research participants. A purposive sampling approach was used to conduct the qualitative interviews during March-June 2018. Qualitative data were analysed using a content analysis and inductive approach.

RESULTS

Responses from 24 (46%) CTUs identified that database-screening tools were the most widely used digital tool for recruitment, with the majority being considered effective. The reason (and to whom) these tools were considered effective was in identifying potential participants (for both Site staff and CTU staff) and reaching recruitment target (for CTU staff/CI). Fewer retention tools were used, with short message service (SMS) or email reminders to participants being the most reported. The qualitative interviews revealed five themes across all groups: 'security and transparency'; 'inclusivity and engagement'; 'human interaction'; 'obstacles and risks'; and 'potential benefits'. There was a high level of stakeholder acceptance of the use of digital tools to support trials, despite the lack of evidence to support them over more traditional techniques. Certain differences and similarities between stakeholder groups demonstrated the complexity and challenges of using digital tools for recruiting and retaining research participants.

CONCLUSIONS

Our studies identified a range of digital tools in use in recruitment and retention of RCTs, despite the lack of high-quality evidence to support their use. Understanding the type of digital tools in use to support recruitment and retention will help to inform funders and the wider research community about their value and relevance for future RCTs. Consideration of further focused digital tool reviews and primary research will help to reduce gaps in the evidence base.

EMR-integrated minimal core dataset for routine health care and multiple research settings: A case study for neuroinflammatory demyelinating diseases

Although routine health care and clinical trials usually require the documentation of similar information, data collection is performed independently from each other, resulting in redundant documentation efforts. Standardizing routine documentation can enable secondary use for medical research. Neuroinflammatory demyelinating diseases (NIDs) represent a heterogeneous group of diseases requiring further research to improve patient management. The aim of this work is to develop, implement and evaluate a minimal core dataset in routine health care with a focus on secondary use as case study for NIDs. Therefore, a draft minimal core dataset for NIDs was created by analyzing routine, clinical trial, registry, biobank documentation and existing data standards for NIDs. Data elements (DEs) were converted into the standard format Operational Data Model, semantically annotated and analyzed via frequency analysis. The analysis produced 1958 DEs based on 864 distinct medical concepts. After review and finalization by an interdisciplinary team of neurologists, epidemiologists and medical computer scientists, the minimal core dataset (NID CDEs) consists of 46 common DEs capturing disease-specific information for reuse in the discharge letter and other research settings. It covers the areas of diagnosis, laboratory results, disease progress, expanded disability status scale, therapy and magnetic resonance imaging findings. NID CDEs was implemented in two German university hospitals and a usability study in clinical routine was conducted (participants n = 16) showing a good usability (Mean SUS = 75). From May 2017 to February 2018, 755 patients were documented with the NID CDEs, which indicates the feasibility of developing a minimal core dataset for structured documentation based on previously used documentation standards and integrating the dataset into clinical routine. By sharing, translating and reusing the minimal dataset, a transnational harmonized documentation of patients with NIDs might be realized, supporting interoperability in medical research.

Methodological description of clinical research data collection through electronic medical records in a center participating in an international multicenter study

Data collection for clinical research can be difficult, and electronic health record systems can facilitate this process. The aim of this study was to describe and evaluate the secondary use of electronic health records in data collection for an observational clinical study. We used Cerner Millennium^®, an electronic health record software, following these steps: (1) data crossing between the study’s case report forms and the electronic health record; (2) development of a manual collection method for data not recorded in Cerner Millennium^®; (3) development of a study interface for automatic data collection in the electronic health records; (4) employee training; (5) data quality assessment; and (6) filling out the electronic case report form at the end of the study. Three case report forms were consolidated into the electronic case report form at the end of the study. Researchers performed daily qualitative and quantitative analyses of the data. Data were collected from 94 patients. In the first case report form, 76.5% of variables were obtained electronically, in the second, 95.5%, and in the third, 100%. The daily quality assessment of the whole process showed complete and correct data, widespread employee compliance and minimal interference in their practice. The secondary use of electronic health records is safe and effective, reduces manual labor, and provides data reliability. Anesthetic care and data collection may be done by the same professional.

Connecting healthcare and clinical research: Workflow optimizations through seamless integration of EHR, pseudonymization services and EDC systems

OBJECTIVE

In the last years, several projects promote the secondary use of routine healthcare data based on electronic health record (EHR) data. In multicenter studies, dedicated pseudonymization services are applied for unified pseudonym handling. Healthcare, clinical research and pseudonymization systems are generally disconnected. Hence, the aim of this research work is to integrate these applications and to evaluate the workflow of clinical research.

METHODS

We analyzed and identified technical solutions for legislation compliant automatic pseudonym generation and for the integration into EHR as well as electronic data capture (EDC) systems. The Mainzelliste was used as pseudonymization service, which is available as open source solution and compliant with the data privacy concept in Germany. Subject of the integration was the local EHR and an in-house developed EDC system. A time and motion study was conducted to evaluate the effects on the workflow.

RESULTS

Integration of EHR, pseudonymization service and EDC systems is technically feasible and leads to a less fragmented usage of all applications. Generated pseudonyms are obtained from the service hosted at a trusted third party and can now be used in the EDC as well as in the EHR system for direct access and re-identification. The evaluation of 90 registration iterations shows that the time for documentation has been significantly reduced in average by 39.6 s (56.3%) from 71 ± 8 s to 31 ± 5 s per registered study patient.

CONCLUSIONS

By incorporating EHR, EDC and pseudonymization systems, it is now feasible to support multicenter studies and registers out of an integrated system landscape within a hospital. Optimizing the workflow of patient registration for clinical research allows reduction of double data entry and transcription errors as well as a seamless transition from clinical routine to research data collection.

Effectiveness and cost of recruiting healthy volunteers for clinical research studies using an electronic patient portal: A randomized study

Introduction

It is not clear how to effectively recruit healthy research volunteers.

Methods

We developed an electronic health record (EHR)-based algorithm to identify healthy subjects, who were randomly assigned to receive an invitation to join a research registry via the EHR's patient portal, letters, or phone calls. A follow-up survey assessed contact preferences.

Results

The EHR algorithm accurately identified 858 healthy subjects. Recruitment rates were low, but occurred more quickly via the EHR patient portal than letters or phone calls (2.7 vs. 19.3 or 10.4 d). Effort and costs per enrolled subject were lower for the EHR patient portal (3.0 vs. 17.3 or 13.6 h, $113 vs. $559 or $435). Most healthy subjects indicated a preference for contact via electronic methods.

Conclusions

Healthy subjects can be accurately identified from EHR data, and it is faster and more cost-effective to recruit healthy research volunteers using an EHR patient portal.

Integrated Data Repository Toolkit (IDRT)

Background: In recent years, research data warehouses moved increasingly into the focus of interest of medical research. Nevertheless, there are only a few center-independent infrastructure solutions available. They aim to provide a consolidated view on medical data from various sources such as clinical trials, electronic health records, epidemiological registries or longitudinal cohorts. The i2b2 framework is a well-established solution for such repositories, but it lacks support for importing and integrating clinical data and metadata.

Objectives: The goal of this project was to develop a platform for easy integration and administration of data from heterogeneous sources, to provide capabilities for linking them to medical terminologies and to allow for transforming and mapping of data streams for user-specific views.

Methods: A suite of three tools has been developed: the i2b2 Wizard for simplifying administration of i2b2, the IDRT Import and Mapping Tool for loading clinical data from various formats like CSV, SQL, CDISC ODM or biobanks and the IDRT i2b2 Web Client Plugin for advanced export options. The Import and Mapping Tool also includes an ontology editor for rearranging and mapping patient data and structures as well as annotating clinical data with medical terminologies, primarily those used in Germany (ICD-10-GM, OPS, ICD-O, etc.).

Results: With the three tools functional, new i2b2-based research projects can be created, populated and customized to researcher’s needs in a few hours. Amalgamating data and metadata from different databases can be managed easily. With regards to data privacy a pseudonymization service can be plugged in. Using common ontologies and reference terminologies rather than project-specific ones leads to a consistent understanding of the data semantics.

Conclusions: i2b2’s promise is to enable clinical researchers to devise and test new hypothesis even without a deep knowledge in statistical programing. The approach pre -sented here has been tested in a number of scenarios with millions of observations and tens of thousands of patients. Initially mostly observant, trained researchers were able to construct new analyses on their own. Early feedback indicates that timely and extensive access to their “own” data is appreciated most, but it is also lowering the barrier for other tasks, for instance checking data quality and completeness (missing data, wrong coding).

Clinical Research Informatics: Recent Advances and Future Directions

OBJECTIVES

To summarize significant developments in Clinical Research Informatics (CRI) over the past two years and discuss future directions.

METHODS

Survey of advances, open problems and opportunities in this field based on exploration of current literature.

RESULTS

Recent advances are structured according to three use cases of clinical research: Protocol feasibility, patient identification/ recruitment and clinical trial execution.

DISCUSSION

CRI is an evolving, dynamic field of research. Global collaboration, open metadata, content standards with semantics and computable eligibility criteria are key success factors for future developments in CRI.

Feasibility of automatic evaluation of clinical rules in general practice

PURPOSE

To assess the extent to which clinical rules (CRs) can be implemented for automatic evaluation of quality of care in general practice.

METHODS

We assessed 81 clinical rules (CRs) adapted from a subset of Assessing Care of Vulnerable Elders (ACOVE) clinical rules, against Dutch College of General Practitioners (NHG) data model. Each CR was analyzed using the Logical Elements Rule METHOD: (LERM). LERM is a stepwise method of assessing and formalizing clinical rules for decision support. Clinical rules that satisfied the criteria outlined in the LERM method were judged to be implementable in automatic evaluation in general practice.

RESULTS

Thirty-three out of 81 (40.7%) Dutch-translated ACOVE clinical rules can be automatically evaluated in electronic medical record systems. Seven out of 7 CRs (100%) in the domain of diabetes can be automatically evaluated, 9/17 (52.9%) in medication use, 5/10 (50%) in depression care, 3/6 (50%) in nutrition care, 6/13 (46.1%) in dementia care, 1/6 (16.6%) in end of life care, 2/13 (15.3%) in continuity of care, and 0/9 (0%) in the fall-related care. Lack of documentation of care activities between primary and secondary health facilities and ambiguous formulation of clinical rules were the main reasons for the inability to automate the clinical rules.

CONCLUSION

Approximately two-fifths of the primary care Dutch ACOVE-based clinical rules can be automatically evaluated. Clear definition of clinical rules, improved GP database design and electronic linkage of primary and secondary healthcare facilities can improve prospects of automatic assessment of quality of care. These findings are relevant especially because the Netherlands has very high automation of primary care.

Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned

Background

The development of Electronic Health Records (EHRs) in hospitals offers the ability to reuse data from patient care activities for clinical research. EHR4CR is a European public-private partnership aiming to develop a computerized platform that enables the re-use of data collected from EHRs over its network. However, the reproducibility of queries may depend on attributes of the local data. Our objective was 1/ to describe the different steps that were achieved in order to use the EHR4CR platform and 2/ to identify the specific issues that could impact the final performance of the platform.

Methods

We selected three institutional studies covering various medical domains. The studies included a total of 67 inclusion and exclusion criteria and ran in two University Hospitals. We described the steps required to use the EHR4CR platform for a feasibility study. We also defined metrics to assess each of the steps (including criteria complexity, normalization quality, and data completeness of EHRs).

Results

We identified 114 distinct medical concepts from a total of 67 eligibility criteria Among the 114 concepts: 23 (20.2%) corresponded to non-structured data (i.e. for which transformation is needed before analysis), 92 (81%) could be mapped to terminologies used in EHR4CR, and 86 (75%) could be mapped to local terminologies. We identified 51 computable criteria following the normalization process. The normalization was considered by experts to be satisfactory or higher for 64.2% (43/67) of the computable criteria. All of the computable criteria could be expressed using the EHR4CR platform.

Conclusions

We identified a set of issues that could affect the future results of the platform: (a) the normalization of free-text criteria, (b) the translation into computer-friendly criteria and (c) issues related to the execution of the query to clinical data warehouses. We developed and evaluated metrics to better describe the platforms and their result. These metrics could be used for future reports of Clinical Trial Recruitment Support Systems assessment studies, and provide experts and readers with tools to insure the quality of constructed dataset.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0299-3) contains supplementary material, which is available to authorized users.

Electronic Support for Retrospective Analysis in the Field of Radiation Oncology: Proof of Principle Using an Example of Fractionated Stereotactic Radiotherapy of 251 Meningioma Patients

Introduction

The purpose of this study is to verify the possible benefit of a clinical data warehouse (DWH) for retrospective analysis in the field of radiation oncology.

Material and methods

We manually and electronically (using DWH) evaluated demographic, radiotherapy, and outcome data from 251 meningioma patients, who were irradiated from January 2002 to January 2015 at the Department of Radiation Oncology of the Erlangen University Hospital. Furthermore, we linked the Oncology Information System (OIS) MOSAIQ^® to the DWH in order to gain access to irradiation data. We compared the manual and electronic data retrieval method in terms of congruence of data, corresponding time, and personal requirements (physician, physicist, scientific associate).

Results

The electronically supported data retrieval (DWH) showed an average of 93.9% correct data and significantly (p = 0.009) better result compared to manual data retrieval (91.2%). Utilizing a DWH enables the user to replace large amounts of manual activities (668 h), offers the ability to significantly reduce data collection time and labor demand (35 h), while simultaneously improving data quality. In our case, work time for manually data retrieval was 637 h for the scientific assistant, 26 h for the medical physicist, and 5 h for the physician (total 668 h).

Conclusion

Our study shows that a DWH is particularly useful for retrospective analysis in the radiation oncology field. Routine clinical data for a large patient group can be provided ready for analysis to the scientist and data collection time can be significantly reduced. Furthermore, linking multiple data sources in a DWH offers the ability to improve data quality for retrospective analysis, and future research can be simplified.

Common data elements for secondary use of electronic health record data for clinical trial execution and serious adverse event reporting

BACKGROUND

Data capture is one of the most expensive phases during the conduct of a clinical trial and the increasing use of electronic health records (EHR) offers significant savings to clinical research. To facilitate these secondary uses of routinely collected patient data, it is beneficial to know what data elements are captured in clinical trials. Therefore our aim here is to determine the most commonly used data elements in clinical trials and their availability in hospital EHR systems.

METHODS

Case report forms for 23 clinical trials in differing disease areas were analyzed. Through an iterative and consensus-based process of medical informatics professionals from academia and trial experts from the European pharmaceutical industry, data elements were compiled for all disease areas and with special focus on the reporting of adverse events. Afterwards, data elements were identified and statistics acquired from hospital sites providing data to the EHR4CR project.

RESULTS

The analysis identified 133 unique data elements. Fifty elements were congruent with a published data inventory for patient recruitment and 83 new elements were identified for clinical trial execution, including adverse event reporting. Demographic and laboratory elements lead the list of available elements in hospitals EHR systems. For the reporting of serious adverse events only very few elements could be identified in the patient records.

CONCLUSIONS

Common data elements in clinical trials have been identified and their availability in hospital systems elucidated. Several elements, often those related to reimbursement, are frequently available whereas more specialized elements are ranked at the bottom of the data inventory list. Hospitals that want to obtain the benefits of reusing data for research from their EHR are now able to prioritize their efforts based on this common data element list.

Data Linkage from Clinical to Study Databases via an R Data Warehouse User Interface. Experiences from a Large Clinical Follow-up Study

BACKGROUND

Data that needs to be documented for clinical studies has often been acquired and documented in clinical routine. Usually this data is manually transferred to Case Report Forms (CRF) and/or directly into an electronic data capture (EDC) system.

OBJECTIVES

To enhance the documentation process of a large clinical follow-up study targeting patients admitted for acutely decompensated heart failure by accessing the data created during routine and study visits from a hospital information system (HIS) and by transferring it via a data warehouse (DWH) into the study's EDC system.

METHODS

This project is based on the clinical DWH developed at the University of Würzburg. The DWH was extended by several new data domains including data created by the study team itself. An R user interface was developed for the DWH that allows to access its source data in all its detail, to transform data as comprehensively as possible by R into study-specific variables and to support the creation of data and catalog tables.

RESULTS

A data flow was established that starts with labeling patients as study patients within the HIS and proceeds with updating the DWH with this label and further data domains at a daily rate. Several study-specific variables were defined using the implemented R user interface of the DWH. This system was then used to export these variables as data tables ready for import into our EDC system. The data tables were then used to initialize the first 296 patients within the EDC system by pseudonym, visit and data values. Afterwards, these records were filled with clinical data on heart failure, vital parameters and time spent on selected wards.

CONCLUSIONS

This solution focuses on the comprehensive access and transformation of data for a DWH-EDC system linkage. Using this system in a large clinical study has demonstrated the feasibility of this approach for a study with a complex visit schedule.

Using Electronic Health Records to Support Clinical Trials: A Report on Stakeholder Engagement for EHR4CR

Background. The conduct of clinical trials is increasingly challenging due to greater complexity and governance requirements as well as difficulties with recruitment and retention. Electronic Health Records for Clinical Research (EHR4CR) aims at improving the conduct of trials by using existing routinely collected data, but little is known about stakeholder views on data availability, information governance, and acceptable working practices. Methods. Senior figures in healthcare organisations across Europe were provided with a description of the project and structured interviews were subsequently conducted to elicit their views. Results. 37 structured interviewees in Germany, UK, Switzerland, and France indicated strong support for the proposed EHR4CR platform. All interviewees reported that using the platform for assessing feasibility would enhance the conduct of clinical trials and the majority also felt it would reduce workloads. Interviewees felt the platform could enhance trial recruitment and adverse event reporting but also felt it could raise either ethical or information governance concerns in their country. Conclusions. There was clear support for EHR4CR and a belief that it could reduce workloads and improve the conduct and quality of trials. However data security, privacy, and information governance issues would need to be carefully managed in the development of the platform.

Real-Time Enrollment Dashboard For Multisite Clinical Trials

OBJECTIVE

Achieving patient recruitment goals are critical for the successful completion of a clinical trial. We designed and developed a web-based dashboard for assisting in the management of clinical trial screening and enrollment.

MATERIALS AND METHODS

We use the dashboard to assist in the management of two observational studies of community-acquired pneumonia. Clinical research associates and managers using the dashboard were surveyed to determine its effectiveness as compared with traditional direct communication.

RESULTS

The dashboard has been in use since it was first introduced in May of 2014. Of the 23 staff responding to the survey, 77% felt that it was easier or much easier to use the dashboard for communication than to use direct communication.

CONCLUSION

We have designed and implemented a visualization dashboard for managing multi-site clinical trial enrollment in two community acquired pneumonia studies. Information dashboards are a useful tool for clinical trial management. They can be used as a standalone trial information tool or included into a larger management system.

Ontology-based data integration between clinical and research systems

Data from the electronic medical record comprise numerous structured but uncoded ele-ments, which are not linked to standard terminologies. Reuse of such data for secondary research purposes has gained in importance recently. However, the identification of rele-vant data elements and the creation of database jobs for extraction, transformation and loading (ETL) are challenging: With current methods such as data warehousing, it is not feasible to efficiently maintain and reuse semantically complex data extraction and trans-formation routines. We present an ontology-supported approach to overcome this challenge by making use of abstraction: Instead of defining ETL procedures at the database level, we use ontologies to organize and describe the medical concepts of both the source system and the target system. Instead of using unique, specifically developed SQL statements or ETL jobs, we define declarative transformation rules within ontologies and illustrate how these constructs can then be used to automatically generate SQL code to perform the desired ETL procedures. This demonstrates how a suitable level of abstraction may not only aid the interpretation of clinical data, but can also foster the reutilization of methods for un-locking it.

Information technology for clinical, translational and comparative effectiveness research. Findings from the section clinical research informatics

OBJECTIVE

To select and summarize key contributions to current research in the field of Clinical Research Informatics (CRI).

METHOD

A bibliographic search using a combination of MeSH and free terms search over PubMed was performed followed by a blinded review.

RESULTS

The review process resulted in the selection of four papers illustrating various aspects of current research efforts in the area of CRI. The first paper tackles the challenge of extracting accurate phenotypes from Electronic Healthcare Records (EHRs). Privacy protection within shared de-identified, patient-level research databases is the focus of the second selected paper. Two other papers exemplify the growing role of formal representation of clinical data - in metadata repositories - and knowledge - in ontologies - for supporting the process of reusing data for clinical research.

CONCLUSIONS

The selected articles demonstrate how concrete platforms are currently achieving interoperability across clinical research and care domains and have reached the evaluation phase. When EHRs linked to genetic data have the potential to shift the research focus from research driven patient recruitment to phenotyping in large population, a key issue is to lower patient re-identification risks for biomedical research databases. Current research illustrates the potential of knowledge engineering to support, in the coming years, the scientific lifecycle of clinical research.

A methodology for a minimum data set for rare diseases to support national centers of excellence for healthcare and research

BACKGROUND

Although rare disease patients make up approximately 6-8% of all patients in Europe, it is often difficult to find the necessary expertise for diagnosis and care and the patient numbers needed for rare disease research. The second French National Plan for Rare Diseases highlighted the necessity for better care coordination and epidemiology for rare diseases. A clinical data standard for normalization and exchange of rare disease patient data was proposed. The original methodology used to build the French national minimum data set (F-MDS-RD) common to the 131 expert rare disease centers is presented.

METHODS

To encourage consensus at a national level for homogeneous data collection at the point of care for rare disease patients, we first identified four national expert groups. We reviewed the scientific literature for rare disease common data elements (CDEs) in order to build the first version of the F-MDS-RD. The French rare disease expert centers validated the data elements (DEs). The resulting F-MDS-RD was reviewed and approved by the National Plan Strategic Committee. It was then represented in an HL7 electronic format to maximize interoperability with electronic health records.

RESULTS

The F-MDS-RD is composed of 58 DEs in six categories: patient, family history, encounter, condition, medication, and questionnaire. It is HL7 compatible and can use various ontologies for diagnosis or sign encoding. The F-MDS-RD was aligned with other CDE initiatives for rare diseases, thus facilitating potential interconnections between rare disease registries.

CONCLUSIONS

The French F-MDS-RD was defined through national consensus. It can foster better care coordination and facilitate determining rare disease patients' eligibility for research studies, trials, or cohorts. Since other countries will need to develop their own standards for rare disease data collection, they might benefit from the methods presented here.

Employing computers for the recruitment into clinical trials: a comprehensive systematic review

BACKGROUND

Medical progress depends on the evaluation of new diagnostic and therapeutic interventions within clinical trials. Clinical trial recruitment support systems (CTRSS) aim to improve the recruitment process in terms of effectiveness and efficiency.

OBJECTIVE

The goals were to (1) create an overview of all CTRSS reported until the end of 2013, (2) find and describe similarities in design, (3) theorize on the reasons for different approaches, and (4) examine whether projects were able to illustrate the impact of CTRSS.

METHODS

We searched PubMed titles, abstracts, and keywords for terms related to CTRSS research. Query results were classified according to clinical context, workflow integration, knowledge and data sources, reasoning algorithm, and outcome.

RESULTS

A total of 101 papers on 79 different systems were found. Most lacked details in one or more categories. There were 3 different CTRSS that dominated: (1) systems for the retrospective identification of trial participants based on existing clinical data, typically through Structured Query Language (SQL) queries on relational databases, (2) systems that monitored the appearance of a key event of an existing health information technology component in which the occurrence of the event caused a comprehensive eligibility test for a patient or was directly communicated to the researcher, and (3) independent systems that required a user to enter patient data into an interface to trigger an eligibility assessment. Although the treating physician was required to act for the patient in older systems, it is now becoming increasingly popular to offer this possibility directly to the patient.

CONCLUSIONS

Many CTRSS are designed to fit the existing infrastructure of a clinical care provider or the particularities of a trial. We conclude that the success of a CTRSS depends more on its successful workflow integration than on sophisticated reasoning and data processing algorithms. Furthermore, some of the most recent literature suggest that an increase in recruited patients and improvements in recruitment efficiency can be expected, although the former will depend on the error rate of the recruitment process being replaced. Finally, to increase the quality of future CTRSS reports, we propose a checklist of items that should be included.

The correlation between the number of eligible patients in routine clinical practice and the low recruitment level in clinical trials: a retrospective study using electronic medical records

BACKGROUND

A number of clinical trials have encountered difficulties enrolling a sufficient number of patients upon initiating the trial. Recently, many screening systems that search clinical data warehouses for patients who are eligible for clinical trials have been developed. We aimed to estimate the number of eligible patients using routine electronic medical records (EMRs) and to predict the difficulty of enrolling sufficient patients prior to beginning a trial.

METHODS

Investigator-initiated clinical trials that were conducted at Kyoto University Hospital between July 2004 and January 2011 were included in this study. We searched the EMRs for eligible patients and calculated the eligible EMR patient index by dividing the number of eligible patients in the EMRs by the target sample size. Additionally, we divided the trial eligibility criteria into corresponding data elements in the EMRs to evaluate the completeness of mapping clinical manifestation in trial eligibility criteria into structured data elements in the EMRs. We evaluated the correlation between the index and the accrual achievement with Spearman's rank correlation coefficient.

RESULTS

Thirteen of 19 trials did not achieve their original target sample size. Overall, 55% of the trial eligibility criteria were mapped into data elements in EMRs. The accrual achievement demonstrated a significant positive correlation with the eligible EMR patient index (r = 0.67, 95% confidence interval (CI), 0.42 to 0.92). The receiver operating characteristic analysis revealed an eligible EMR patient index cut-off value of 1.7, with a sensitivity of 69.2% and a specificity of 100.0%.

CONCLUSIONS

Our study suggests that the eligible EMR patient index remains exploratory but could be a useful component of the feasibility study when planning a clinical trial. Establishing a step to check whether there are likely to be a sufficient number of eligible patients enables sponsors and investigators to concentrate their resources and efforts on more achievable trials.

Evaluating predictive modeling algorithms to assess patient eligibility for clinical trials from routine data

BACKGROUND

The necessity to translate eligibility criteria from free text into decision rules that are compatible with data from the electronic health record (EHR) constitutes the main challenge when developing and deploying clinical trial recruitment support systems. Recruitment decisions based on case-based reasoning, i.e. using past cases rather than explicit rules, could dispense with the need for translating eligibility criteria and could also be implemented largely independently from the terminology of the EHR's database. We evaluated the feasibility of predictive modeling to assess the eligibility of patients for clinical trials and report on a prototype's performance for different system configurations.

METHODS

The prototype worked by using existing basic patient data of manually assessed eligible and ineligible patients to induce prediction models. Performance was measured retrospectively for three clinical trials by plotting receiver operating characteristic curves and comparing the area under the curve (ROC-AUC) for different prediction algorithms, different sizes of the learning set and different numbers and aggregation levels of the patient attributes.

RESULTS

Random forests were generally among the best performing models with a maximum ROC-AUC of 0.81 (CI: 0.72-0.88) for trial A, 0.96 (CI: 0.95-0.97) for trial B and 0.99 (CI: 0.98-0.99) for trial C. The full potential of this algorithm was reached after learning from approximately 200 manually screened patients (eligible and ineligible). Neither block- nor category-level aggregation of diagnosis and procedure codes influenced the algorithms' performance substantially.

CONCLUSIONS

Our results indicate that predictive modeling is a feasible approach to support patient recruitment into clinical trials. Its major advantages over the commonly applied rule-based systems are its independency from the concrete representation of eligibility criteria and EHR data and its potential for automation.