Employing computers for the recruitment into clinical trials: a comprehensive systematic review

Model of integrated mental health video consultations for people with depression or anxiety in primary care (PROVIDE-C): assessor masked, multicentre, randomised controlled trial

OBJECTIVE

To evaluate whether an integrated mental health video consultation approach (PROVIDE model) can improve symptoms compared with usual care in adults with depression and anxiety disorders attending primary care.

DESIGN

Assessor masked, multicentre, randomised controlled trial (PROVIDE-C).

SETTING

In 29 primary care practices in Germany, working remotely online from one trial hub.

PARTICIPANTS

376 adults (18-81 years) who presented to their general practitioner (GP) with depression or anxiety, or both.

INTERVENTION

Participants were randomised (1:1) to receive the PROVIDE model (n=187) or usual care (n=189). Usual care was provided by GPs through interventions such as brief counselling and psychotropic medication prescriptions and may or may not have included referrals to mental health specialists. The PROVIDE model comprised transdiagnostic treatment provided through five real-time video sessions between the patient at the primary care practice and a mental health specialist at an offsite location.

MAIN OUTCOME MEASURES

The primary outcome was the absolute change in the mean severity of depressive and anxiety symptoms measured using the patient health questionnaire anxiety and depression scale (PHQ-ADS) at six months, in the intention-to-treat population. Secondary outcomes, measured at six and 12 months, included PHQ-ADS subscores, psychological distress related to somatic symptoms, recovery, health related quality of life, quality and patient centredness of chronic illness care, and adverse events.

RESULTS

Between 24 March 2020 and 23 November 2021, 376 patients were randomised into treatment groups. Mean age was 45 years (standard deviation (SD) 14), 63% of the participants were female, and mean PHQ-ADS-score was 26 points (SD 7.6). Compared with usual care, the PROVIDE intervention led to improvements in severity of depressive and anxiety symptom (adjusted mean change difference in the PHQ-ADS score -2.4 points (95% confidence interval -4.5 to -0.4), P=0.02) at six months. The effects were sustained at 12 months (-2.9 (-5.0 to -0.7), P<0.01). No serious adverse events were reported in either group.

CONCLUSIONS

Through relatively low intensity treatment, the PROVIDE model led to a decrease in depressive and anxiety symptoms with small effects in the short and long term. Depression and anxiety disorders are prevalent and therefore the small effect might cumulatively impact on population health in this population.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04316572.

recruIT: A cloud-native clinical trial recruitment support system based on Health Level 7 Fast Healthcare Interoperability Resources (HL7 FHIR) and the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM)

BACKGROUND

Clinical trials (CTs) are foundational to the advancement of evidence-based medicine and recruiting a sufficient number of participants is one of the crucial steps to their successful conduct. Yet, poor recruitment remains the most frequent reason for premature discontinuation or costly extension of clinical trials.

METHODS

We designed and implemented a novel, open-source software system to support the recruitment process in clinical trials by generating automatic recruitment recommendations. The development is guided by modern, cloud-native design principles and based on Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) as an interoperability standard with the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) being used as a source of patient data. We evaluated the usability using the system usability scale (SUS) after deploying the application for use by study personnel.

RESULTS

The implementation is based on the OMOP CDM as a repository of patient data that is continuously queried for possible trial candidates based on given clinical trial eligibility criteria. A web-based screening list can be used to display the candidates and email notifications about possible new trial participants can be sent automatically. All interactions between services use HL7 FHIR as the communication standard. The system can be installed using standard container technology and supports more sophisticated deployments on Kubernetes clusters. End-users (n = 19) rated the system with a SUS score of 79.9/100.

CONCLUSION

We contribute a novel, open-source implementation to support the patient recruitment process in clinical trials that can be deployed using state-of-the art technologies. According to the SUS score, the system provides good usability.

Sociotechnical feasibility of natural language processing-driven tools in clinical trial eligibility prescreening for Alzheimer's disease and related dementias

BACKGROUND

Alzheimer's disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants.

OBJECTIVE

This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools' cognitive complexity's effect on usability to identify cognitive support strategies.

METHODS

A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models.

RESULTS

Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability.

DISCUSSION

Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools' usability and efficacy in clinical research prescreening.

CONCLUSION

User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.

Effectiveness of IT-supported patient recruitment: study protocol for an interrupted time series study at ten German university hospitals

BACKGROUND

As part of the German Medical Informatics Initiative, the MIRACUM project establishes data integration centers across ten German university hospitals. The embedded MIRACUM Use Case "Alerting in Care - IT Support for Patient Recruitment", aims to support the recruitment into clinical trials by automatically querying the repositories for patients satisfying eligibility criteria and presenting them as screening candidates. The objective of this study is to investigate whether the developed recruitment tool has a positive effect on study recruitment within a multi-center environment by increasing the number of participants. Its secondary objective is the measurement of organizational burden and user satisfaction of the provided IT solution.

METHODS

The study uses an Interrupted Time Series Design with a duration of 15 months. All trials start in the control phase of randomized length with regular recruitment and change to the intervention phase with additional IT support. The intervention consists of the application of a recruitment-support system which uses patient data collected in general care for screening according to specific criteria. The inclusion and exclusion criteria of all selected trials are translated into a machine-readable format using the OHDSI ATLAS tool. All patient data from the data integration centers is regularly checked against these criteria. The primary outcome is the number of participants recruited per trial and week standardized by the targeted number of participants per week and the expected recruitment duration of the specific trial. Secondary outcomes are usability, usefulness, and efficacy of the recruitment support. Sample size calculation based on simple parallel group assumption can demonstrate an effect size of d=0.57 on a significance level of 5% and a power of 80% with a total number of 100 trials (10 per site). Data describing the included trials and the recruitment process is collected at each site. The primary analysis will be conducted using linear mixed models with the actual recruitment number per week and trial standardized by the expected recruitment number per week and trial as the dependent variable.

DISCUSSION

The application of an IT-supported recruitment solution developed in the MIRACUM consortium leads to an increased number of recruited participants in studies at German university hospitals. It supports employees engaged in the recruitment of trial participants and is easy to integrate in their daily work.

Applying Natural Language Processing to Textual Data From Clinical Data Warehouses: Systematic Review

BACKGROUND

In recent years, health data collected during the clinical care process have been often repurposed for secondary use through clinical data warehouses (CDWs), which interconnect disparate data from different sources. A large amount of information of high clinical value is stored in unstructured text format. Natural language processing (NLP), which implements algorithms that can operate on massive unstructured textual data, has the potential to structure the data and make clinical information more accessible.

OBJECTIVE

The aim of this review was to provide an overview of studies applying NLP to textual data from CDWs. It focuses on identifying the (1) NLP tasks applied to data from CDWs and (2) NLP methods used to tackle these tasks.

METHODS

This review was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched for relevant articles in 3 bibliographic databases: PubMed, Google Scholar, and ACL Anthology. We reviewed the titles and abstracts and included articles according to the following inclusion criteria: (1) focus on NLP applied to textual data from CDWs, (2) articles published between 1995 and 2021, and (3) written in English.

RESULTS

We identified 1353 articles, of which 194 (14.34%) met the inclusion criteria. Among all identified NLP tasks in the included papers, information extraction from clinical text (112/194, 57.7%) and the identification of patients (51/194, 26.3%) were the most frequent tasks. To address the various tasks, symbolic methods were the most common NLP methods (124/232, 53.4%), showing that some tasks can be partially achieved with classical NLP techniques, such as regular expressions or pattern matching that exploit specialized lexica, such as drug lists and terminologies. Machine learning (70/232, 30.2%) and deep learning (38/232, 16.4%) have been increasingly used in recent years, including the most recent approaches based on transformers. NLP methods were mostly applied to English language data (153/194, 78.9%).

CONCLUSIONS

CDWs are central to the secondary use of clinical texts for research purposes. Although the use of NLP on data from CDWs is growing, there remain challenges in this field, especially with regard to languages other than English. Clinical NLP is an effective strategy for accessing, extracting, and transforming data from CDWs. Information retrieved with NLP can assist in clinical research and have an impact on clinical practice.

Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study

Objectives

To efficiently assess the disease-modifying potential of new osteoarthritis treatments, clinical trials need progression-enriched patient populations. To assess whether the application of machine learning results in patient selection enrichment, we developed a machine learning recruitment strategy targeting progressive patients and validated it in the IMI-APPROACH knee osteoarthritis prospective study.

Design

We designed a two-stage recruitment process supported by machine learning models trained to rank candidates by the likelihood of progression. First stage models used data from pre-existing cohorts to select patients for a screening visit. The second stage model used screening data to inform the final inclusion. The effectiveness of this process was evaluated using the actual 24-month progression.

Results

From 3500 candidate patients, 433 with knee osteoarthritis were screened, 297 were enrolled, and 247 completed the 2-year follow-up visit. We observed progression related to pain (P, 30%), structure (S, 13%), and combined pain and structure (P ​+ ​S, 5%), and a proportion of non-progressors (N, 52%) ∼15% lower vs an unenriched population. Our model predicted these outcomes with AUC of 0.86 [95% CI, 0.81-0.90] for pain-related progression and AUC of 0.61 [95% CI, 0.52-0.70] for structure-related progression. Progressors were ranked higher than non-progressors for P ​+ ​S (median rank 65 vs 143, AUC = 0.75), P (median rank 77 vs 143, AUC = 0.71), and S patients (median rank 107 vs 143, AUC = 0.57).

Conclusions

The machine learning-supported recruitment resulted in enriched selection of progressive patients. Further research is needed to improve structural progression prediction and assess this strategy in an interventional trial.

Effects of Communication Source and Racial Representation in Clinical Trial Recruitment Flyers

The current study was designed to examine effective message strategies that can be employed in designing mediated communication messages to improve clinical trial research participation. In the study, a total of 300 participants completed an online experiment in which they responded to five different clinical trial recruitment advertisements whose information sources varied in their credentials and race. Overall, peer-featured ads in which previous clinical trial participants communicated their prior experience in clinical trial participation, compared to expert-featured ads in which medical doctors communicated information about clinical trials, led to higher message and topic relevance, higher message credibility, more favorable attitudes toward clinical trials, and higher intentions to participate in future clinical trials. Further, there was a statistically significant interaction among source credentials, racial match (between source and participant), and participant's race on message and topic relevance such that both White and Black participants rated ads from racially mismatched peers highly effective (greater message and topic relevance); however, for doctor featured ads, White participants reported higher message and topic relevance for racially matched (White doctor) ads, and Black participants reported higher message and topic relevance for racially mismatched (White doctor) ads. We discuss theoretical and practical implications.

ELaPro, a LOINC-mapped core dataset for top laboratory procedures of eligibility screening for clinical trials

Background

Screening for eligible patients continues to pose a great challenge for many clinical trials. This has led to a rapidly growing interest in standardizing computable representations of eligibility criteria (EC) in order to develop tools that leverage data from electronic health record (EHR) systems. Although laboratory procedures (LP) represent a common entity of EC that is readily available and retrievable from EHR systems, there is a lack of interoperable data models for this entity of EC. A public, specialized data model that utilizes international, widely-adopted terminology for LP, e.g. Logical Observation Identifiers Names and Codes (LOINC®), is much needed to support automated screening tools.

Objective

The aim of this study is to establish a core dataset for LP most frequently requested to recruit patients for clinical trials using LOINC terminology. Employing such a core dataset could enhance the interface between study feasibility platforms and EHR systems and significantly improve automatic patient recruitment.

Methods

We used a semi-automated approach to analyze 10,516 screening forms from the Medical Data Models (MDM) portal’s data repository that are pre-annotated with Unified Medical Language System (UMLS). An automated semantic analysis based on concept frequency is followed by an extensive manual expert review performed by physicians to analyze complex recruitment-relevant concepts not amenable to automatic approach.

Results

Based on analysis of 138,225 EC from 10,516 screening forms, 55 laboratory procedures represented 77.87% of all UMLS laboratory concept occurrences identified in the selected EC forms. We identified 26,413 unique UMLS concepts from 118 UMLS semantic types and covered the vast majority of Medical Subject Headings (MeSH) disease domains.

Conclusions

Only a small set of common LP covers the majority of laboratory concepts in screening EC forms which supports the feasibility of establishing a focused core dataset for LP. We present ELaPro, a novel, LOINC-mapped, core dataset for the most frequent 55 LP requested in screening for clinical trials. ELaPro is available in multiple machine-readable data formats like CSV, ODM and HL7 FHIR. The extensive manual curation of this large number of free-text EC as well as the combining of UMLS and LOINC terminologies distinguishes this specialized dataset from previous relevant datasets in the literature.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01611-y.

Using Technology to Enhance Cancer Clinical Trial Participation

The COVID-19 pandemic presented many challenges to health care systems, including oncology clinical research programs. There were substantial negative effects on oncology clinical trial screening, enrollment, and study activities that forced institutions and regulatory bodies to develop innovative solutions to maintain robust and equitable participation in these trials. Digital pathology innovations at Memorial Sloan Kettering Cancer Center have streamlined the diagnostic life cycle for patients with cancer, and the seamless integration of digital pathology services with next-generation sequencing and other molecular pathology services have accelerated the time to diagnosis and receipt of molecular results. Timely access to these results, coupled with Memorial Sloan Kettering Cancer Center's knowledge engine OncoKB, enhances patient clinical trial coordination precisely and efficiently. At the Sarah Cannon Research Institute, centralized remote clinical trial matching and screening, virtual molecular tumor boards, and centralized molecular interpretation support services have empowered clinic staff to identify more efficiently potential participants in clinical research, despite the COVID-19 pandemic. In addition, the U.S. Food and Drug Administration Oncology Center of Excellence has been involved in several efforts to address challenges for patients with cancer during the COVID-19 pandemic, including writing guidance documents and participating in efforts to modernize clinical trials. The enclosed personal experience of a patient with cancer currently participating in an oncology clinical trial emphasizes the need for continued decreasing of barriers to study participation. Clinical trial advances that were accelerated by the pandemic will ultimately help patients with cancer and the greater oncology health care community.

Patient Recruitment System for Clinical Trials: Mixed Methods Study About Requirements at Ten University Hospitals

BACKGROUND

Clinical trials are the gold standard for advancing medical knowledge and improving patient outcomes. For their success, an appropriately sized cohort is required. However, patient recruitment remains one of the most challenging aspects of clinical trials. Information technology (IT) support systems-for instance, patient recruitment systems-may help overcome existing challenges and improve recruitment rates, when customized to the user needs and environment.

OBJECTIVE

The goal of our study is to describe the status quo of patient recruitment processes and to identify user requirements for the development of a patient recruitment system.

METHODS

We conducted a web-based survey with 56 participants as well as semistructured interviews with 33 participants from 10 German university hospitals.

RESULTS

We here report the recruitment procedures and challenges of 10 university hospitals. The recruitment process was influenced by diverse factors such as the ward, use of software, and the study inclusion criteria. Overall, clinical staff seemed more involved in patient identification, while the research staff focused on screening tasks. Ad hoc and planned screenings were common. Identifying eligible patients was still associated with significant manual efforts. The recruitment staff used Microsoft Office suite because tailored software were not available. To implement such software, data from disparate sources will need to be made available. We discussed concrete technical challenges concerning patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration, and we contributed to the support of developing a successful system.

CONCLUSIONS

Identifying eligible patients is still associated with significant manual efforts. To fully make use of the high potential of IT in patient recruitment, many technical and process challenges have to be solved first. We contribute and discuss concrete technical challenges for patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration.

Fast Healthcare Interoperability Resources (FHIR) for Interoperability in Health Research: A Systematic Review (Preprint)

Background

The standard Fast Healthcare Interoperability Resources (FHIR) is widely used in health information technology. However, its use as a standard for health research is still less prevalent. To use existing data sources more efficiently for health research, data interoperability becomes increasingly important. FHIR provides solutions by offering resource domains such as “Public Health & Research” and “Evidence-Based Medicine” while using already established web technologies. Therefore, FHIR could help standardize data across different data sources and improve interoperability in health research.

Objective

The aim of our study was to provide a systematic review of existing literature and determine the current state of FHIR implementations in health research and possible future directions.

Methods

We searched the PubMed/MEDLINE, Embase, Web of Science, IEEE Xplore, and Cochrane Library databases for studies published from 2011 to 2022. Studies investigating the use of FHIR in health research were included. Articles published before 2011, abstracts, reviews, editorials, and expert opinions were excluded. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and registered this study with PROSPERO (CRD42021235393). Data synthesis was done in tables and figures.

Results

We identified a total of 998 studies, of which 49 studies were eligible for inclusion. Of the 49 studies, most (73%, n=36) covered the domain of clinical research, whereas the remaining studies focused on public health or epidemiology (6%, n=3) or did not specify their research domain (20%, n=10). Studies used FHIR for data capture (29%, n=14), standardization of data (41%, n=20), analysis (12%, n=6), recruitment (14%, n=7), and consent management (4%, n=2). Most (55%, 27/49) of the studies had a generic approach, and 55% (12/22) of the studies focusing on specific medical specialties (infectious disease, genomics, oncology, environmental health, imaging, and pulmonary hypertension) reported their solutions to be conferrable to other use cases. Most (63%, 31/49) of the studies reported using additional data models or terminologies: Systematized Nomenclature of Medicine Clinical Terms (29%, n=14), Logical Observation Identifiers Names and Codes (37%, n=18), International Classification of Diseases 10th Revision (18%, n=9), Observational Medical Outcomes Partnership common data model (12%, n=6), and others (43%, n=21). Only 4 (8%) studies used a FHIR resource from the domain “Public Health & Research.” Limitations using FHIR included the possible change in the content of FHIR resources, safety, legal matters, and the need for a FHIR server.

Conclusions

Our review found that FHIR can be implemented in health research, and the areas of application are broad and generalizable in most use cases. The implementation of international terminologies was common, and other standards such as the Observational Medical Outcomes Partnership common data model could be used as a complement to FHIR. Limitations such as the change of FHIR content, lack of FHIR implementation, safety, and legal matters need to be addressed in future releases to expand the use of FHIR and, therefore, interoperability in health research.

A systematic review on natural language processing systems for eligibility prescreening in clinical research

OBJECTIVE

We conducted a systematic review to assess the effect of natural language processing (NLP) systems in improving the accuracy and efficiency of eligibility prescreening during the clinical research recruitment process.

MATERIALS AND METHODS

Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews, a protocol for study eligibility was developed a priori and registered in the PROSPERO database. Using predetermined inclusion criteria, studies published from database inception through February 2021 were identified from 5 databases. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was adapted to determine the study quality and the risk of bias of the included articles.

RESULTS

Eleven studies representing 8 unique NLP systems met the inclusion criteria. These studies demonstrated moderate study quality and exhibited heterogeneity in the study design, setting, and intervention type. All 11 studies evaluated the NLP system's performance for identifying eligible participants; 7 studies evaluated the system's impact on time efficiency; 4 studies evaluated the system's impact on workload; and 2 studies evaluated the system's impact on recruitment.

DISCUSSION

NLP systems in clinical research eligibility prescreening are an understudied but promising field that requires further research to assess its impact on real-world adoption. Future studies should be centered on continuing to develop and evaluate relevant NLP systems to improve enrollment into clinical studies.

CONCLUSION

Understanding the role of NLP systems in improving eligibility prescreening is critical to the advancement of clinical research recruitment.

What Oncologists Want: Identifying Challenges and Preferences on Diagnosis Data Entry to Reduce EHR-Induced Burden and Improve Clinical Data Quality

PURPOSE

Accurate recording of diagnosis (DX) data in electronic health records (EHRs) is important for clinical practice and learning health care. Previous studies show statistically stable patterns of data entry in EHRs that contribute to inaccurate DX, likely because of a lack of data entry support. We conducted qualitative research to characterize the preferences of oncological care providers on cancer DX data entry in EHRs during clinical practice.

METHODS

We conducted semistructured interviews and focus groups to uncover common themes on DX data entry preferences and barriers to accurate DX recording. Then, we developed a survey questionnaire sent to a cohort of oncologists to verify the generalizability of our initial findings. We constrained our participants to a single specialty and institution to ensure similar clinical backgrounds and clinical experience with a single EHR system.

RESULTS

A total of 12 neuro-oncologists and thoracic oncologists were involved in the interviews and focus groups. The survey developed from these two initial thrusts was distributed to 19 participants yielding a 94.7% survey response rate. Clinicians reported similar user interface experiences, barriers, and dissatisfaction with current DX entry systems including repetitive entry operations, difficulty in finding specific DX options, time-consuming interactions, and the need for workarounds to maintain efficiency. The survey revealed inefficient DX search interfaces and challenging entry processes as core barriers.

CONCLUSION

Oncologists seem to be divided between specific DX data entry and time efficiency because of current interfaces and feel hindered by the burdensome and repetitive nature of EHR data entry. Oncologists' top concern for adopting data entry support interventions is ensuring that it provides significant time-saving benefits and increasing workflow efficiency. Future interventions should account for time efficiency, beyond ensuring data entry effectiveness.

ACE: the Advanced Cohort Engine for searching longitudinal patient records

OBJECTIVE

To propose a paradigm for a scalable time-aware clinical data search, and to describe the design, implementation and use of a search engine realizing this paradigm.

MATERIALS AND METHODS

The Advanced Cohort Engine (ACE) uses a temporal query language and in-memory datastore of patient objects to provide a fast, scalable, and expressive time-aware search. ACE accepts data in the Observational Medicine Outcomes Partnership Common Data Model, and is configurable to balance performance with compute cost. ACE's temporal query language supports automatic query expansion using clinical knowledge graphs. The ACE API can be used with R, Python, Java, HTTP, and a Web UI.

RESULTS

ACE offers an expressive query language for complex temporal search across many clinical data types with multiple output options. ACE enables electronic phenotyping and cohort-building with subsecond response times in searching the data of millions of patients for a variety of use cases.

DISCUSSION

ACE enables fast, time-aware search using a patient object-centric datastore, thereby overcoming many technical and design shortcomings of relational algebra-based querying. Integrating electronic phenotype development with cohort-building enables a variety of high-value uses for a learning health system. Tradeoffs include the need to learn a new query language and the technical setup burden.

CONCLUSION

ACE is a tool that combines a unique query language for time-aware search of longitudinal patient records with a patient object datastore for rapid electronic phenotyping, cohort extraction, and exploratory data analyses.

Baseline characteristics and comparability of older multimorbid patients with polypharmacy and general practitioners participating in a randomized controlled primary care trial

Objectives

Recruiting general practitioners (GPs) and their multimorbid older patients for trials is challenging for multiple reasons (e.g., high workload, limited mobility). The comparability of study participants is important for interpreting study findings. This manuscript describes the baseline characteristics of GPs and patients participating in the ‘Optimizing PharmacoTherapy in older multimorbid adults In primary CAre’ (OPTICA) trial, a study of optimization of pharmacotherapy for multimorbid older adults. The overall aim of this study was to determine if the GPs and patients participating in the OPTICA trial are comparable to the real-world population in Swiss primary care.

Design

Analysis of baseline data from GPs and patients in the OPTICA trial and a reference cohort from the FIRE (‘Family medicine ICPC Research using Electronic medical records’) project.

Setting

Primary care, Switzerland.

Participants

Three hundred twenty-three multimorbid (≥ 3 chronic conditions) patients with polypharmacy (≥ 5 regular medications) aged ≥ 65 years and 43 GPs recruited for the OPTICA trial were compared to 22,907 older multimorbid patients with polypharmacy and 227 GPs from the FIRE database.

Methods

We compared the characteristics of GPs and patients participating in the OPTICA trial with other GPs and other older multimorbid adults with polypharmacy in the FIRE database. We described the baseline willingness to have medications deprescribed of the patients participating in the OPTICA trial using the revised Patients’ Attitudes Towards Deprescribing (rPATD) questionnaire.

Results

The GPs in the FIRE project and OPTICA were similar in terms of sociodemographic characteristics and their work as a GP (e.g. aged in their fifties, ≥ 10 years of experience, ≥ 60% are self-employed, ≥ 80% work in a group practice). The median age of patients in the OPTICA trial was 77 years and 45% of trial participants were women. Patients participating in the OPTICA trial and patients in the FIRE database were comparable in terms of age, certain clinical characteristics (e.g. systolic blood pressure, body mass index) and health services use (e.g. selected lab and vital data measurements). More than 80% of older multimorbid patients reported to be willing to stop ≥ 1 of their medications if their doctor said that this would be possible.

Conclusion

The characteristics of patients and GPs recruited into the OPTICA trial are relatively comparable to characteristics of a real-world Swiss population, which indicates that recruiting a generalizable patient sample is possible in the primary care setting. Multimorbid patients in the OPTICA trial reported a high willingness to have medications deprescribed.

Trial registration

Clinicaltrials.gov (NCT03724539), KOFAM (Swiss national portal) (SNCTP000003060), Universal Trial Number (U1111-1226-8013)

Supplementary Information

The online version contains supplementary material available at 10.1186/s12875-021-01488-8.

Shared-Task Worklists Improve Clinical Trial Recruitment Workflow in an Academic Emergency Department

BACKGROUND

Clinical trials performed in our emergency department at Barnes-Jewish Hospital utilize a centralized infrastructure for alerting, screening, and enrollment with rule-based alerts sent to clinical research coordinators. Previously, all alerts were delivered as text messages via dedicated cellular phones. As the number of ongoing clinical trials increased, the volume of alerts grew to an unmanageable level. Therefore, we have changed our primary notification delivery method to study-specific, shared-task worklists integrated with our pre-existing web-based screening documentation system.

OBJECTIVE

To evaluate the effects on screening and recruitment workflow of replacing text-message delivery of clinical trial alerts with study-specific shared-task worklists in a high-volume academic emergency department supporting multiple concurrent clinical trials.

METHODS

We analyzed retrospective data on alerting, screening, and enrollment for 10 active clinical trials pre- and postimplementation of shared-task worklists.

RESULTS

Notifications signaling the presence of potentially eligible subjects for clinical trials were more likely to result in a screen (p < 0.001) with the implementation of shared-task worklists compared with notifications delivered as text messages for 8/10 clinical trials. The change in workflow did not alter the likelihood of a notification resulting in an enrollment (p = 0.473). The Director of Research reported a substantial reduction in the amount of time spent redirecting clinical research coordinator screening activities.

CONCLUSION

Shared-task worklists, with the functionalities we have described, offer a viable alternative to delivery of clinical trial alerts via text message directly to clinical research coordinators recruiting for multiple concurrent clinical trials in a high-volume academic emergency department.

Electronic medical record-based cohort selection and direct-to-patient, targeted recruitment: early efficacy and lessons learned

OBJECTIVE

The study sought to characterize institution-wide participation in secure messaging (SM) at a large academic health network, describe our experience with electronic medical record (EMR)-based cohort selection, and discuss the potential roles of SM for research recruitment.

MATERIALS AND METHODS

Study teams defined eligibility criteria to create a computable phenotype, structured EMR data, to identify and recruit participants. Patients with SM accounts matching this phenotype received recruitment messages. We compared demographic characteristics across SM users and the overall health system. We also tabulated SM activation and use, characteristics of individual studies, and efficacy of the recruitment methods.

RESULTS

Of the 1 308 820 patients in the health network, 40% had active SM accounts. SM users had a greater proportion of white and non-Hispanic patients than nonactive SM users id. Among the studies included (n = 13), 77% recruited participants with a specific disease or condition. All studies used demographic criteria for their phenotype, while 46% (n = 6) used demographic, disease, and healthcare utilization criteria. The average SM response rate was 2.9%, with higher rates among condition-specific (3.4%) vs general health (1.4%) studies. Those studies with a more inclusive comprehensive phenotype had a higher response rate.

DISCUSSION

Target population and EMR queries (computable phenotypes) affect recruitment efficacy and should be considered when designing an EMR-based recruitment strategy.

CONCLUSIONS

SM guided by EMR-based cohort selection is a promising approach to identify and enroll research participants. Efforts to increase the number of active SM users and response rate should be implemented to enhance the effectiveness of this recruitment strategy.

Catch Me if You Can: Acute Events Hidden in Structured Chronic Disease Diagnosis Descriptions Show Detectable Recording Patterns in EHR

Our previous research shows that structured cancer DX description data accuracy varied across electronic health record (EHR) segments (e.g. encounter DX, problem list, etc.). We provide initial evidence corroborating these findings in EHRs from patients with diabetes. We hypothesized that the odds of recording an "uncontrolled diabetes" DX increased after a hemoglobin A1c result above 9% and that this rate would vary across EHR segments. Our statistical models revealed that each DX indicating uncontrolled diabetes was 2.6% more likely to occur post-A1c>9% overall (adj-p=.0005) and 3.9% after controlling for EHR segment (adj-p<.0001). However, odds ratios varied across segments (1.021 Collapse

Key Words Collapse

MESH Headings

• Chronic Disease
• Datasets as Topic
• Diabetes Mellitus/diagnosis
• Electronic Health Records
• Glycated Hemoglobin/analysis
• Humans
• Machine Learning
• Medical Informatics/methods
• Models, Statistical
• Odds Ratio

Collapse

Grants

• UL1 TR001420 NCATS NIH HHS

Collapse

Affiliation(s)

Franck Diaz-Garelli University of North Carolina at Charlotte. Charlotte, NC
Kristin M Lenoir Wake Forest School of Medicine, Winston Salem, NC
Brian J Wells Wake Forest School of Medicine, Winston Salem, NC

Collapse

Electronic Health Record Algorithm Development for Research Subject Recruitment Using Colonoscopy Appointment Scheduling

INTRODUCTION

Electronic health records (EHRs) are often leveraged in medical research to recruit study participants efficiently. The purpose of this study was to validate and refine the logic of an EHR algorithm for identifying potentially eligible participants for a comparative effectiveness study of fecal immunochemical tests (FITs), using colonoscopy as the standard.

METHODS

An Epic report was built to identify patients who met the eligibility criteria to recruit patients having a screening or surveillance colonoscopy. With the goal of maximizing the number of potentially eligible patients that could be recruited, researchers, with the assistance of information technology and scheduling staff, developed the algorithm for identifying potential subjects in the EHR. Two validation methods, descriptive statistics and manual verification, were used.

RESULTS

The algorithm was refined over 3 iterations leading to the following criteria being used for generating the report: Age, Appointment Made On/Cancel Date, Appointment Procedure, Contact Type, Date Range, Encounter Departments, ICD-10 codes, and Patient Type. Appointment Serial Number/Contact Serial Number were output fields that allowed the tracking of cancellations and reschedules.

CONCLUSION

Development of an EHR algorithm saved time in that most individuals ineligible for the study were excluded before patient medical record review. Running daily reports that included cancellations and rescheduled appointments allowed for maximum recruitment in a time frame appropriate for the use of the FITs. This algorithm demonstrates that refining the algorithm iteratively and adding cancellations and reschedules of colonoscopies increased the accuracy of reaching all potential patients for recruitment.

Heregulin (HRG) assessment for clinical trial eligibility testing in a molecular registry (PRAEGNANT) in Germany

BACKGROUND

Eligibility criteria are a critical part of clinical trials, as they define the patient population under investigation. Besides certain patient characteristics, clinical trials often include biomarker testing for eligibility. However, patient-identification mostly relies on the trial site itself and is often a time-consuming procedure, which could result in missing out on potentially eligible patients. Pre-selection of those patients using a registry could facilitate the process of eligibility testing and increase the number of identified patients. One aim with the PRAEGNANT registry (NCT02338167) is to identify patients for therapies based on clinical and molecular data. Here, we report eligibility testing for the SHERBOC trial using the German PRAEGNANT registry.

METHODS

Heregulin (HRG) has been reported to identify patients with better responses to therapy with the anti-HER3 monoclonal antibody seribantumab (MM-121). The SHERBOC trial investigated adding seribantumab (MM-121) to standard therapy in patients with advanced HER2-negative, hormone receptor-positive (HR-positive) breast cancer and HRG overexpression. The PRAEGNANT registry was used for identification and tumor testing, helping to link potential HRG positive patients to the trial. Patients enrolled in PRAEGNANT have invasive and metastatic or locally advanced, inoperable breast cancer. Patients eligible for SHERBOC were identified by using the registry. Study aims were to describe the HRG positivity rate, screening procedures, and patient characteristics associated with inclusion and exclusion criteria.

RESULTS

Among 2769 unselected advanced breast cancer patients, 650 were HER2-negative, HR-positive and currently receiving first- or second-line treatment, thus potentially eligible for SHERBOC at the end of current treatment; 125 patients also met further clinical eligibility criteria (e.g. menopausal status, ECOG). In the first/second treatment lines, patients selected for SHERBOC based on further eligibility criteria had a more favorable prognosis than those not selected. HRG status was tested in 38 patients, 14 of whom (36.8%) proved to be HRG-positive.

CONCLUSION

Using a real-world breast cancer registry allowed identification of potentially eligible patients for SHERBOC focusing on patients with HER3 overexpressing, HR-positive, HER2-negative metastatic breast cancer. This approach may provide insights into differences between patients eligible or non-eligible for clinical trials.

TRIAL REGISTRATION

Clinicaltrials, NCT02338167 , Registered 14 January 2015 - retrospectively registered.

Key components and IT assistance of participant management in clinical research: a scoping review

Objectives

Managing participants and their data are fundamental for the success of a clinical trial. Our review identifies and describes processes that deal with management of trial participants and highlights information technology (IT) assistance for clinical research in the context of participant management.

Methods

A scoping literature review design, based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement, was used to identify literature on trial participant-related proceedings, work procedures, or workflows, and assisting electronic systems.

Results

The literature search identified 1329 articles of which 111 were included for analysis. Participant-related procedures were categorized into 4 major trial processes: recruitment, obtaining informed consent, managing identities, and managing administrative data. Our results demonstrated that management of trial participants is considered in nearly every step of clinical trials, and that IT was successfully introduced to all participant-related areas of a clinical trial to facilitate processes.

Discussion

There is no precise definition of participant management, so a broad search strategy was necessary, resulting in a high number of articles that had to be excluded. Nevertheless, this review provides a comprehensive overview of participant management-related components, which was lacking so far. The review contributes to a better understanding of how computer-assisted management of participants in clinical trials is possible.

EHR phenotyping for research recruitment: Researcher, IRB, and physician perspectives on approaches to contacting patients

Introduction:

Failure to achieve accrual goals is a common problem in health-related research. Electronic health records represent a promising resource, offering the ability to identify a precisely defined cohort of patients who meet inclusion/exclusion criteria. However, challenges associated with the recruitment process remain and institutional policies vary.

Methods:

We interviewed researchers, institutional review board chairs, and primary care physicians in North Carolina and Tennessee. Questions focused on strategies for initiating contact with potentially eligible patients, as well as recruitment letters asking recipients to opt in versus opt out of further communication.

Results:

When we asked about initiating contact with prospective participants, qualitative themes included trust, credibility, and established relationships; research efficiency and validity; privacy and autonomy; the intersection between research and clinical care; and disruption to physician–researcher and physician–patient relationships. All interviewees said it was acceptable for researchers to contact patients through their physicians; most said it was acceptable for researchers to contact patients directly. Over half chose contact through physicians as more appropriate. Regarding recruitment letters, qualitative themes included the quality of the participant pool; privacy and control; research efficiency and representativeness; and patients’ opportunity to make their own decisions. All interviewees said asking recipients to opt in to further communication was acceptable; nearly all said opt out was acceptable. Similar proportions chose each approach as more appropriate.

Conclusions:

Comparing these results to our previous research with patients reveals potential differences in stakeholder perspectives. We offer suggestions for developing balanced approaches that respect patients and facilitate the advancement of science.

A review of big data and medical research

Universally, the volume of data has increased, with the collection rate doubling every 40 months, since the 1980s. "Big data" is a term that was introduced in the 1990s to include data sets too large to be used with common software. Medicine is a major field predicted to increase the use of big data in 2025. Big data in medicine may be used by commercial, academic, government, and public sectors. It includes biologic, biometric, and electronic health data. Examples of biologic data include biobanks; biometric data may have individual wellness data from devices; electronic health data include the medical record; and other data demographics and images. Big data has also contributed to the changes in the research methodology. Changes in the clinical research paradigm has been fueled by large-scale biological data harvesting (biobanks), which is developed, analyzed, and managed by cheaper computing technology (big data), supported by greater flexibility in study design (real-world data) and the relationships between industry, government regulators, and academics. Cultural changes along with easy access to information via the Internet facilitate ease of participation by more people. Current needs demand quick answers which may be supplied by big data, biobanks, and changes in flexibility in study design. Big data can reveal health patterns, and promises to provide solutions that have previously been out of society's grasp; however, the murkiness of international laws, questions of data ownership, public ignorance, and privacy and security concerns are slowing down the progress that could otherwise be achieved by the use of big data. The goal of this descriptive review is to create awareness of the ramifications for big data and to encourage readers that this trend is positive and will likely lead to better clinical solutions, but, caution must be exercised to reduce harm.

Applied Practice and Possible Leverage Points for Information Technology Support for Patient Screening in Clinical Trials: Qualitative Study

BACKGROUND

Clinical trials are one of the most challenging and meaningful designs in medical research. One essential step before starting a clinical trial is screening, that is, to identify patients who fulfill the inclusion criteria and do not fulfill the exclusion criteria. The screening step for clinical trials might be supported by modern information technology (IT).

OBJECTIVE

This explorative study aimed (1) to obtain insights into which tools for feasibility estimations and patient screening are actually used in clinical routine and (2) to determine which method and type of IT support could benefit clinical staff.

METHODS

Semistandardized interviews were conducted in 5 wards (cardiology, gynecology, gastroenterology, nephrology, and palliative care) in a German university hospital. Of the 5 interviewees, 4 were directly involved in patient screening. Three of them were clinicians, 1 was a study nurse, and 1 was a research assistant.

RESULTS

The existing state of study feasibility estimation and the screening procedure were dominated by human communication and estimations from memory, although there were many possibilities for IT support. Success mostly depended on the experience and personal motivation of the clinical staff. Electronic support has been used but with little importance so far. Searches in ward-specific patient registers (databases) and searches in clinical information systems were reported. Furthermore, free-text searches in medical reports were mentioned. For potential future applications, a preference for either proactive or passive systems was not expressed. Most of the interviewees saw the potential for the improvement of the actual systems, but they were also largely satisfied with the outcomes of the current approach. Most of the interviewees were interested in learning more about the various ways in which IT could support and relieve them in their clinical routine.

CONCLUSIONS

Overall, IT support currently plays a minor role in the screening step for clinical trials. The lack of IT usage and the estimations made from memory reported by all the participants might constrain cognitive resources, which might distract from clinical routine. We conclude that electronic support for the screening step for clinical trials is still a challenge and that education of the staff about the possibilities for electronic support in clinical trials is necessary.

Digital tools for the recruitment and retention of participants in randomised controlled trials: a systematic map

BACKGROUND

Recruiting and retaining participants in randomised controlled trials (RCTs) is challenging. Digital tools, such as social media, data mining, email or text-messaging, could improve recruitment or retention, but an overview of this research area is lacking. We aimed to systematically map the characteristics of digital recruitment and retention tools for RCTs, and the features of the comparative studies that have evaluated the effectiveness of these tools during the past 10 years.

METHODS

We searched Medline, Embase, other databases, the Internet, and relevant web sites in July 2018 to identify comparative studies of digital tools for recruiting and/or retaining participants in health RCTs. Two reviewers independently screened references against protocol-specified eligibility criteria. Included studies were coded by one reviewer with 20% checked by a second reviewer, using pre-defined keywords to describe characteristics of the studies, populations and digital tools evaluated.

RESULTS

We identified 9163 potentially relevant references, of which 104 articles reporting 105 comparative studies were included in the systematic map. The number of published studies on digital tools has doubled in the past decade, but most studies evaluated digital tools for recruitment rather than retention. The key health areas investigated were health promotion, cancers, circulatory system diseases and mental health. Few studies focussed on minority or under-served populations, and most studies were observational. The most frequently-studied digital tools were social media, Internet sites, email and tv/radio for recruitment; and email and text-messaging for retention. One quarter of the studies measured efficiency (cost per recruited or retained participant) but few studies have evaluated people's attitudes towards the use of digital tools.

CONCLUSIONS

This systematic map highlights a number of evidence gaps and may help stakeholders to identify and prioritise further research needs. In particular, there is a need for rigorous research on the efficiency of the digital tools and their impact on RCT participants and investigators, perhaps as studies-within-a-trial (SWAT) research. There is also a need for research into how digital tools may improve participant retention in RCTs which is currently underrepresented relative to recruitment research.

REGISTRATION

Not registered; based on a pre-specified protocol, peer-reviewed by the project's Advisory Board.

Workflow Differences Affect Data Accuracy in Oncologic EHRs: A First Step Toward Detangling the Diagnosis Data Babel

PURPOSE

Diagnosis (DX) information is key to clinical data reuse, yet accessible structured DX data often lack accuracy. Previous research hints at workflow differences in cancer DX entry, but their link to clinical data quality is unclear. We hypothesized that there is a statistically significant relationship between workflow-describing variables and DX data quality.

METHODS

We extracted DX data from encounter and order tables within our electronic health records (EHRs) for a cohort of patients with confirmed brain neoplasms. We built and optimized logistic regressions to predict the odds of fully accurate (ie, correct neoplasm type and anatomic site), inaccurate, and suboptimal (ie, vague) DX entry across clinical workflows. We selected our variables based on correlation strength of each outcome variable.

RESULTS

Both workflow and personnel variables were predictive of DX data quality. For example, a DX entered in departments other than oncology had up to 2.89 times higher odds of being accurate (P < .0001) compared with an oncology department; an outpatient care location had up to 98% fewer odds of being inaccurate (P < .0001), but had 458 times higher odds of being suboptimal (P < .0001) compared with main campus, including the cancer center; and a DX recoded by a physician assistant had 85% fewer odds of being suboptimal (P = .005) compared with those entered by physicians.

CONCLUSION

These results suggest that differences across clinical workflows and the clinical personnel producing EHR data affect clinical data quality. They also suggest that the need for specific structured DX data recording varies across clinical workflows and may be dependent on clinical information needs. Clinicians and researchers reusing oncologic data should consider such heterogeneity when conducting secondary analyses of EHR data.

Using digital tools in the recruitment and retention in randomised controlled trials: survey of UK Clinical Trial Units and a qualitative study

BACKGROUND

Recruitment and retention of participants in randomised controlled trials (RCTs) is a key determinant of success but is challenging. Trialists and UK Clinical Research Collaboration (UKCRC) Clinical Trials Units (CTUs) are increasingly exploring the use of digital tools to identify, recruit and retain participants. The aim of this UK National Institute for Health Research (NIHR) study was to identify what digital tools are currently used by CTUs and understand the performance characteristics required to be judged useful.

METHODS

A scoping of searches (and a survey with NIHR funding staff), a survey with all 52 UKCRC CTUs and 16 qualitative interviews were conducted with five stakeholder groups including trialists within CTUs, funders and research participants. A purposive sampling approach was used to conduct the qualitative interviews during March-June 2018. Qualitative data were analysed using a content analysis and inductive approach.

RESULTS

Responses from 24 (46%) CTUs identified that database-screening tools were the most widely used digital tool for recruitment, with the majority being considered effective. The reason (and to whom) these tools were considered effective was in identifying potential participants (for both Site staff and CTU staff) and reaching recruitment target (for CTU staff/CI). Fewer retention tools were used, with short message service (SMS) or email reminders to participants being the most reported. The qualitative interviews revealed five themes across all groups: 'security and transparency'; 'inclusivity and engagement'; 'human interaction'; 'obstacles and risks'; and 'potential benefits'. There was a high level of stakeholder acceptance of the use of digital tools to support trials, despite the lack of evidence to support them over more traditional techniques. Certain differences and similarities between stakeholder groups demonstrated the complexity and challenges of using digital tools for recruiting and retaining research participants.

CONCLUSIONS

Our studies identified a range of digital tools in use in recruitment and retention of RCTs, despite the lack of high-quality evidence to support their use. Understanding the type of digital tools in use to support recruitment and retention will help to inform funders and the wider research community about their value and relevance for future RCTs. Consideration of further focused digital tool reviews and primary research will help to reduce gaps in the evidence base.

A novel approach to conducting clinical trials in the community setting: utilizing patient-driven platforms and social media to drive web-based patient recruitment

Background

Participant recruitment for clinical research studies remains a significant challenge for researchers. Novel approaches to recruitment are necessary to ensure that populations are easier to reach. In the context of rare diseases, social media provides a unique opportunity for connecting with patient groups that have representatively lower diagnosis rates when compared with more common diseases or illness. We describe the implementation of designing a patient-centered approach to message design for the purposes of recruiting patients for clinical research studies for rare disease populations.

Methods

Using an iterative research approach, we analyzed our previous experience of using web-based direct-to-patient recruitment methods to compare these online strategies with traditional center of excellence recruitment strategies. After choosing six research studies for inclusion in the previous study, in-depth, online interviews (n = 37) were conducted with patients represented in each disease category to develop and test recruitment message strategies for social media and a Web-based platform for patients to access study information and pre-screen. Finally, relationships were established with Patient Advocacy Groups representing each rare disease category to ensure further dissemination of recruitment materials via their own social media networks.

Results

Guided by social marketing theory, we created and tested various recruitment message designs. Three key message concepts preferred by patients emerged: (1) infographic; (2) positive emotional messages; and (3) educational information for sharing. A base study website was designed and created based on data from patient interviews. This website includes the option for potential participants to pre-screen and determine their eligibility for the study.

Conclusions

Study participants report wanting to be involved in the design and implementation of recruitment approaches for clinical research studies. The application of the aforementioned methods could aide in the evolution of clinical research practices for the recruitment of both rare and common diseases, where patient-centric approaches can help to create targeted messages designs that participants pre-test and support.

Primary care perspectives on implementation of clinical trial recruitment

Introduction:

Poor clinical trial (CT) recruitment is a significant barrier to translating basic science discoveries into medical practice. Improving support for primary care provider (PCP) referral of patients to CTs may be an important part of the solution. However, implementing CT referral support in primary care is not only technically challenging, but also presents challenges at the person and organization levels.

Methods:

The objectives of this study were (1) to characterize provider and clinical supervisor attitudes and perceptions regarding CT research, recruitment, and referrals in primary care and (2) to identify perceived workflow strategies and facilitators relevant to designing a technology-supported primary care CT referral program. Focus groups were conducted with PCPs, directors, and supervisors.

Results:

Analysis indicated widespread support for the intrinsic scientific value of CTs, while at the same time deep concerns regarding protecting patient well-being, perceived loss of control when patients participate in trials, concern about the impact of point-of-care referrals on clinic workflow, the need for standard processes, and the need for CT information that enables referring providers to quickly confirm that the burdens are justified by the benefits at both patient and provider levels. PCP suggestions pertinent to implementing a CT referral decision support system are reported.

Conclusion:

The results from this work contribute to developing an implementation approach to support increased referral of patients to CTs.

Feasibility analysis of conducting observational studies with the electronic health record

Background

The secondary use of electronic health records (EHRs) promises to facilitate medical research. We reviewed general data requirements in observational studies and analyzed the feasibility of conducting observational studies with structured EHR data, in particular diagnosis and procedure codes.

Methods

After reviewing published observational studies from the University Hospital of Erlangen for general data requirements, we identified three different study populations for the feasibility analysis with eligibility criteria from three exemplary observational studies. For each study population, we evaluated the availability of relevant patient characteristics in our EHR, including outcome and exposure variables. To assess data quality, we computed distributions of relevant patient characteristics from the available structured EHR data and compared them to those of the original studies. We implemented computed phenotypes for patient characteristics where necessary. In random samples, we evaluated how well structured patient characteristics agreed with a gold standard from manually interpreted free texts. We categorized our findings using the four data quality dimensions “completeness”, “correctness”, “currency” and “granularity”.

Results

Reviewing general data requirements, we found that some investigators supplement routine data with questionnaires, interviews and follow-up examinations. We included 847 subjects in the feasibility analysis (Study 1 n = 411, Study 2 n = 423, Study 3 n = 13). All eligibility criteria from two studies were available in structured data, while one study required computed phenotypes in eligibility criteria. In one study, we found that all necessary patient characteristics were documented at least once in either structured or unstructured data. In another study, all exposure and outcome variables were available in structured data, while in the other one unstructured data had to be consulted. The comparison of patient characteristics distributions, as computed from structured data, with those from the original study yielded similar distributions as well as indications of underreporting. We observed violations in all four data quality dimensions.

Conclusions

While we found relevant patient characteristics available in structured EHR data, data quality problems may entail that it remains a case-by-case decision whether diagnosis and procedure codes are sufficient to underpin observational studies. Free-text data or subsequently supplementary study data may be important to complement a comprehensive patient history.

A tale of three subspecialties: Diagnosis recording patterns are internally consistent but Specialty-Dependent

BACKGROUND

Structured diagnosis (DX) are crucial for secondary use of electronic health record (EHR) data. However, they are often suboptimally recorded. Our previous work showed initial evidence of variable DX recording patterns in oncology charts even after biopsy records are available.

OBJECTIVE

We verified this finding's internal and external validity. We hypothesized that this recording pattern would be preserved in a larger cohort of patients for the same disease. We also hypothesized that this effect would vary across subspecialties.

METHODS

We extracted DX data from EHRs of patients treated for brain, lung, and pancreatic neoplasms, identified through clinician-led chart reviews. We used statistical methods (i.e., binomial and mixed model regressions) to test our hypotheses.

RESULTS

We found variable recording patterns in brain neoplasm DX (i.e., larger number of distinct DX-OR = 2.2, P < 0.0001, higher descriptive specificity scores-OR = 1.4, P < 0.0001-and much higher entropy after the BX-OR = 3.8 P = 0.004 and OR = 8.0, P < 0.0001), confirming our initial findings. We also found strikingly different patterns for lung and pancreas DX. Although both seemed to have much lower DX sequence entropy after the BX-OR = 0.198, P = 0.015 and OR = 0.099, P = 0.015, respectively compared to OR = 3.8 P = 0.004). We also found statistically significant differences between the brain dataset and both the lung (P < 0.0001) and pancreas (0.009

CONCLUSION

Our results suggest that disease-specific DX entry patterns exist and are established differently by clinical subspecialty. These differences should be accounted for during clinical data reuse and data quality assessments but also during EHR entry system design to maximize accurate, precise and consistent data entry likelihood.

Collapse

Key Words

• clinical data management
• data quality
• electronic health records
• learning healthcare system
• secondary use of clinical data

Collapse

MESH Headings Collapse

Grants

• K12 GM102773 NIGMS NIH HHS
• UL1 TR001420 NCATS NIH HHS

Collapse

Affiliation(s)

Jose-Franck Diaz-Garelli Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Roy Strowd Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Tamjeed Ahmed Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Brian J Wells Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Rebecca Merrill Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Javier Laurini Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Boris Pasche Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Umit Topaloglu Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA

Collapse

Embedding research recruitment in a community resource e-prescribing system: lessons from an implementation study on Chicago's South Side

OBJECTIVE

The study sought to implement and assess the CommunityRx e-prescribing system to recruit research participants from a predominantly non-Hispanic Black community on Chicago's South Side.

MATERIALS AND METHODS

CommunityRx integrates with electronic medical record systems to generate a personalized list of health-promoting community resources (HealtheRx). Between December 2015 and December 2016, HealtheRxs distributed at outpatient visits to adults with asthma or chronic obstructive pulmonary disease also incentivized participation in a pulmonary research registry. Usual practices for registry recruitment continued in parallel.

RESULTS

Focus groups established acceptability and appropriateness among the target population. Pulmonary research registry recruitment information was included on 13 437 HealtheRxs. Forty-one (90% non-Hispanic Black) patients responded with willingness to participate and 9 (8 non-Hispanic Black) returned a signed consent required to enroll. Usual recruitment practices enrolled 4 registrants (1 non-Hispanic Black).

DISCUSSION

Automating research recruitment using a community e-prescribing system is feasible.

CONCLUSIONS

Implementation of an electronic medical record-integrated, community resource referral tool promotes enrollment of eligible underrepresented research participants; however, enrollment was low.

Lost in Translation: Diagnosis Records Show More Inaccuracies After Biopsy in Oncology Care EHRs

The use of diagnosis (DX) data is crucial to secondary use of electronic health record (EHR) data, yet accessible structured DX data often lack in accuracy. DX descriptions associated with structured DX codes vary even after recording biopsy results; this may indicate poor data quality. We hypothesized that biopsy reports in cancer care charts do not improve intrinsic DX data quality. We analyzed DX data for a manually well-annotated cohort of patients with brain neoplasms. We built statistical models to predict the number of fully-accurate (i.e., correct neoplasm type and anatomical location) and inaccurate DX (i.e. type or location contradicts cohort data) descriptions. We found some evidence of statistically larger numbers of fully-accurate (RR=3.07, p=0.030) but stronger evidence of much larger numbers of inaccurate DX (RR=12.3, p=0.001 and RR=19.6, p<0.0001) after biopsy result recording. Still, 65.9% of all DX records were neither fully-accurate nor fully-inaccurate. These results suggest EHRs must be modified to support more reliable DX data recording and secondary use of EHR data.

Patient perspectives on use of electronic health records for research recruitment

BACKGROUND

EHR phenotyping offers the ability to rapidly assemble a precisely defined cohort of patients prescreened for eligibility to participate in health-related research. Even so, stakeholders in the process must still contend with the practical and ethical challenges associated with research recruitment. Patient perspectives on these matters are particularly important given that the success of research recruitment depends on patients' willingness to participate.

METHODS

We conducted 15 focus groups (n = 110 participants) in four counties in diverse regions of the southeastern US: Appalachia, the Mississippi Delta, and the Piedmont area of North Carolina. Based on a hypothetical study of a behavioral intervention for type 2 diabetes, we asked about the acceptability and appropriateness of direct investigator versus physician-mediated contact with patients for research recruitment, and whether patients should be asked to opt in or opt out of further contact in response to recruitment letters.

RESULTS

For initial contact, nearly all participants said it would be acceptable for researchers to contact patients directly and three-fourths said that it would be acceptable for researchers to contact patients through their physicians. When we asked which would be most appropriate, a substantial majority chose direct contact. Themes that arose in the discussion included trust and transparency, decision-making power, the effect on research, and the effect on patient care. For response expectations, the vast majority of participants said both opt-in and opt-out would be acceptable-typically finding neither especially problematic and noting that both afford patients the opportunity to make their own decisions.

CONCLUSIONS

External validity relies heavily on researchers' success enrolling eligible patients and failure to reach accrual targets is a costly and common barrier to advancing scientific knowledge. Our results suggest that patients recognize multiple advantages and disadvantages of different research recruitment strategies and place value on the implications not just for themselves, but also for researchers and healthcare providers. Our findings, including rich qualitative detail, contribute to the body of empirical and ethical literature on improving research recruitment and suggest specific ways forward as well as important areas for future research.

A review of the impact of utilising electronic medical records for clinical research recruitment

INTRODUCTION

Recruitment is an important aspect of clinical research, as poor recruitment could undermine the scientific value of a trial or delay the development process of new treatments. The development of electronic medical records provides a new way to identify potential participants for trials by matching the eligibility criteria with patients' data within electronic medical records.

METHODS

A literature search was performed to examine the effectiveness and efficiency of the electronic medical record recruitment method using MEDLINE, PubMed, PubMed Central, CINAHL Plus with Full Text, ScienceDirect and Cochrane Library databases. These searches generated 11 articles that met the eligibility criteria, and handsearching reference lists generated two additional articles bringing the total number of articles to 13. These articles were subjected to critical appraisal utilising the Effective Public Health Practice Project tool.

RESULTS

Out of the 13 included articles, 11 provided quantitative data on recruitment effectiveness while seven articles provided quantitative data on recruitment efficiency. The automation in screening and patient identification by using alerts, a notification system, to notify research staff of a potential participant, was observed to contribute to higher recruitment yield and reduced workload due to its specificity on participant screening. The use of electronic medical record alerts was found to be associated with better recruitment outcomes when they were sent to dedicated research staff rather than physicians. Using electronic medical records for recruitment was found to be effective due to its capability for patient identification outside working hours and fast processing time, which was particularly useful for clinical trials in acute conditions. Several challenges may hinder the impact of the electronic medical record recruitment method, including the lack of conformity of clinical trial eligibility criteria and electronic medical record data structure and missing data. 'Alert fatigue' could also impact on the effectiveness of this method in the long term.

CONCLUSION

The results from this review supports electronic medical record being an effective and efficient method for clinical trial recruitment. Recommendations were made in order to maximise the potential of the electronic medical record recruitment method and also for future research in order to improve the quality of evidence to support this strategy for recruitment.

A Practical Do-It-Yourself Recruitment Framework for Concurrent eHealth Clinical Trials: Simple Architecture (Part 1)

Background

The ability to identify, screen, and enroll potential research participants in an efficient and timely manner is crucial to the success of clinical trials. In the age of the internet, researchers can be confronted with large numbers of people contacting the program, overwhelming study staff and frustrating potential participants.

Objective

This paper describes a “do-it-yourself” recruitment support framework (DIY-RSF) that uses tools readily available in many academic research settings to support remote participant recruitment, prescreening, enrollment, and management across multiple concurrent eHealth clinical trials.

Methods

This work was conducted in an academic research center focused on developing and evaluating behavioral intervention technologies. A needs assessment consisting of unstructured individual and group interviews was conducted to identify barriers to recruitment and important features for the new system.

Results

We describe a practical and adaptable recruitment management architecture that used readily available software, such as REDCap (Research Electronic Data Capture) and standard statistical software (eg, SAS, R), to create an automated recruitment framework that supported prescreening potential participants, consent to join a research registry, triaging for management of multiple trials, capture of eligibility information for each phase of a recruitment pipeline, and staff management tools including monitoring of participant flow and task assignment/reassignment features. The DIY-RSF was launched in July 2015. As of July 2017, the DIY-RSF has supported the successful recruitment efforts for eight trials, producing 14,557 participant records in the referral tracking database and 5337 participants in the center research registry. The DIY-RSF has allowed for more efficient use of staff time and more rapid processing of potential applicants.

Conclusions

Using tools already supported at many academic institutions, we describe the architecture and utilization of an adaptable referral management framework to support recruitment for multiple concurrent clinical trials. The DIY-RSF can serve as a guide for leveraging common technologies to improve clinical trial recruitment procedures.

Prescreening clinical trial volunteers using an online personality questionnaire

BACKGROUND

The cost of a clinical trial is affected by the efficiency of participant recruitment. It would be desirable to create a prescreening method that identifies appropriate candidates for full screening, in order to prevent inconvenience for both trial and volunteers. This study presents an online prescreening tool for this purpose.

METHODS

In order to facilitate recruitment of 24 individuals meeting the criteria for generalized anxiety disorder to a pharmacological functional magnetic resonance imaging trial, we created an online personality questionnaire that generated a personality profile for each respondent and screened for the trial's basic criteria.

RESULTS

Our online platform screened 6,293 people for anxious personality traits in 1 year. A total of 862 eligible individuals were identified through this route, each of whom automatically received an email invitation to contact the study team for further telephone screening, if interested. Of those, 266 individuals contacted the team and 173 were telephone screened, with 53 attending the study site for medical checks. Twenty-eight individuals were fully eligible, and 24 completed the trial. This permitted completion on time and on budget.

CONCLUSION

Our online prescreening personality questionnaire platform did not remove the need for telephone screening or onsite medical checks, but increased the efficiency of recruitment through noninvasive identification of those meeting key requirements. Thus, our platform is a useful recruitment technique for clinical trials and is time-saving for both the trial and potential participants.

Development and preliminary evaluation of a patient portal messaging for research recruitment service

Introduction

We developed a service to identify potential study participants through electronic medical records and deliver study invitations through patient portals.

Methods

The service was piloted in a cohort study that used multiple recruitment methods.

Results

Patient portal messages were sent to 1303 individuals and the enrollment rate was 10% (n=127). The patient portal enrollment rate was significantly higher than email and post mail (4%) strategies.

Conclusion

Patient portal messaging was an effective recruitment strategy.

Biopsy Records Do Not Reduce Diagnosis Variability in Cancer Patient EHRs: Are We More Uncertain After Knowing?

Diagnostic codes are crucial for analyses of electronic health record (EHR) data but their accuracy and precision are often lacking. Although providers enter precise diagnoses into progress notes, billing standards may limit the particularity of a diagnostic code. Variability also arises from the creation of multiple descriptions for a particular diagnostic code. We hypothesized that the variability of diagnostic codes would be greater before surgical pathology results were recorded in the medical record. A well annotated cohort of patients with brain neoplasms was studied. After diagnostic pathology reporting, the odds of more distinct diagnostic descriptions were 2.30 times higher (p=0.00358), entropy in diagnostic sequences was 2.26 times higher (p=0.0259) and entropy in diagnostic precision scores was 15.5 times higher (p=0.0324). Although diagnostic codes became more distinct on average after diagnostic pathology reporting, there was a paradoxical increase in the variability of the codes selected. Researchers must be aware of the inconsistencies and variability in particularity in structured diagnostic coding despite the presence of a definitive diagnosis.

Effectiveness and cost of recruiting healthy volunteers for clinical research studies using an electronic patient portal: A randomized study

Introduction

It is not clear how to effectively recruit healthy research volunteers.

Methods

We developed an electronic health record (EHR)-based algorithm to identify healthy subjects, who were randomly assigned to receive an invitation to join a research registry via the EHR's patient portal, letters, or phone calls. A follow-up survey assessed contact preferences.

Results

The EHR algorithm accurately identified 858 healthy subjects. Recruitment rates were low, but occurred more quickly via the EHR patient portal than letters or phone calls (2.7 vs. 19.3 or 10.4 d). Effort and costs per enrolled subject were lower for the EHR patient portal (3.0 vs. 17.3 or 13.6 h, $113 vs. $559 or $435). Most healthy subjects indicated a preference for contact via electronic methods.

Conclusions

Healthy subjects can be accurately identified from EHR data, and it is faster and more cost-effective to recruit healthy research volunteers using an EHR patient portal.

Recruitment methods for survey research: Findings from the Mid-South Clinical Data Research Network

PURPOSE

The objective of this study was to report survey response rates and demographic characteristics of eight recruitment approaches to determine acceptability and effectiveness of large-scale patient recruitment among various populations.

METHODS

We conducted a cross sectional analysis of survey data from two large cohorts. Patients were recruited from the Mid-South Clinical Data Research Network using clinic-based recruitment, research registries, and mail, phone, and email approaches. Response rates are reported as patients who consented for the survey divided by the number of eligible patients approached.

RESULTS

We contacted more than 90,000 patients and 13,197 patients completed surveys. Median age was 56.3years (IQR 40.9, 67.4). Racial/ethnic distribution was 84.1% White, non-Hispanic; 9.9% Black, non-Hispanic; 1.8% Hispanic; and 4.0% other, non-Hispanic. Face-to-face recruitment had the highest response rate of 94.3%, followed by participants who "opted-in" to a registry (76%). The lowest response rate was for unsolicited emails from the clinic (6.1%). Face-to-face recruitment enrolled a higher percentage of participants who self-identified as Black, non-Hispanic compared to other approaches (18.6% face-to-face vs. 8.4% for email).

CONCLUSIONS

Technology-enabled recruitment approaches such as registries and emails are effective for recruiting but may yield less racial/ethnic diversity compared to traditional, more time-intensive approaches.

Clinical Data Reuse or Secondary Use: Current Status and Potential Future Progress

Objective: To perform a review of recent research in clinical data reuse or secondary use, and envision future advances in this field. Methods: The review is based on a large literature search in MEDLINE (through PubMed), conference proceedings, and the ACM Digital Library, focusing only on research published between 2005 and early 2016. Each selected publication was reviewed by the authors, and a structured analysis and summarization of its content was developed. Results: The initial search produced 359 publications, reduced after a manual examination of abstracts and full publications. The following aspects of clinical data reuse are discussed: motivations and challenges, privacy and ethical concerns, data integration and interoperability, data models and terminologies, unstructured data reuse, structured data mining, clinical practice and research integration, and examples of clinical data reuse (quality measurement and learning healthcare systems). Conclusion: Reuse of clinical data is a fast-growing field recognized as essential to realize the potentials for high quality healthcare, improved healthcare management, reduced healthcare costs, population health management, and effective clinical research.

Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned

Background

The development of Electronic Health Records (EHRs) in hospitals offers the ability to reuse data from patient care activities for clinical research. EHR4CR is a European public-private partnership aiming to develop a computerized platform that enables the re-use of data collected from EHRs over its network. However, the reproducibility of queries may depend on attributes of the local data. Our objective was 1/ to describe the different steps that were achieved in order to use the EHR4CR platform and 2/ to identify the specific issues that could impact the final performance of the platform.

Methods

We selected three institutional studies covering various medical domains. The studies included a total of 67 inclusion and exclusion criteria and ran in two University Hospitals. We described the steps required to use the EHR4CR platform for a feasibility study. We also defined metrics to assess each of the steps (including criteria complexity, normalization quality, and data completeness of EHRs).

Results

We identified 114 distinct medical concepts from a total of 67 eligibility criteria Among the 114 concepts: 23 (20.2%) corresponded to non-structured data (i.e. for which transformation is needed before analysis), 92 (81%) could be mapped to terminologies used in EHR4CR, and 86 (75%) could be mapped to local terminologies. We identified 51 computable criteria following the normalization process. The normalization was considered by experts to be satisfactory or higher for 64.2% (43/67) of the computable criteria. All of the computable criteria could be expressed using the EHR4CR platform.

Conclusions

We identified a set of issues that could affect the future results of the platform: (a) the normalization of free-text criteria, (b) the translation into computer-friendly criteria and (c) issues related to the execution of the query to clinical data warehouses. We developed and evaluated metrics to better describe the platforms and their result. These metrics could be used for future reports of Clinical Trial Recruitment Support Systems assessment studies, and provide experts and readers with tools to insure the quality of constructed dataset.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0299-3) contains supplementary material, which is available to authorized users.

Experience With Direct-to-Patient Recruitment for Enrollment Into a Clinical Trial in a Rare Disease: A Web-Based Study

Background

The target sample size for clinical trials often necessitates a multicenter (center of excellence, CoE) approach with associated added complexity, cost, and regulatory requirements. Alternative recruitment strategies need to be tested against this standard model.

Objectives

The aim of our study was to test whether a Web-based direct recruitment approach (patient-centric, PC) using social marketing strategies provides a viable option to the CoE recruitment method.

Methods

PC recruitment and Web-based informed consent was compared with CoE recruitment for a randomized controlled trial (RCT) of continuing versus stopping low-dose prednisone for maintenance of remission of patients with granulomatosis with polyangiitis (GPA).

Results

The PC approach was not as successful as the CoE approach. Enrollment of those confirmed eligible by their physician was 10 of 13 (77%) and 49 of 51 (96%) in the PC and CoE arms, respectively (P=.05). The two approaches were not significantly different in terms of eligibility with 34% of potential participants in the CoE found to be ineligible as compared with 22% in the PC arm (P=.11) nor in provider acceptance, 22% versus 26% (P=.78). There was no difference in the understanding of the trial as reflected in the knowledge surveys of individuals in the PC and CoE arms.

Conclusions

PC recruitment was substantially less successful than that achieved by the CoE approach. However, the PC approach was good at confirming eligibility and was as acceptable to providers and as understandable to patients as the CoE approach. The PC approach should be evaluated in other clinical settings to get a better sense of its potential.

Automatic data source identification for clinical trial eligibility criteria resolution

Clinical trial coordinators refer to both structured and unstructured sources of data when evaluating a subject for eligibility. While some eligibility criteria can be resolved using structured data, some require manual review of clinical notes. An important step in automating the trial screening process is to be able to identify the right data source for resolving each criterion. In this work, we discuss the creation of an eligibility criteria dataset for clinical trials for patients with two disparate diseases, annotated with the preferred data source for each criterion (i.e., structured or unstructured) by annotators with medical training. The dataset includes 50 heart-failure trials with a total of 766 eligibility criteria and 50 trials for chronic lymphocytic leukemia (CLL) with 677 criteria. Further, we developed machine learning models to predict the preferred data source: kernel methods outperform simpler learning models when used with a combination of lexical, syntactic, semantic, and surface features. Evaluation of these models indicates that the performance is consistent across data from both diagnoses, indicating generalizability of our method. Our findings are an important step towards ongoing efforts for automation of clinical trial screening.

Concept and implementation of a study dashboard module for a continuous monitoring of trial recruitment and documentation

BACKGROUND

The difficulty of managing patient recruitment and documentation for clinical trials prompts a demand for instruments for closely monitoring these critical but unpredictable processes. Increasingly adopted Electronic Data Capture (EDC) applications provide novel opportunities to reutilize stored information for an efficient management of traceable trial workflows. In related clinical and administrative settings, so-called digital dashboards that continuously visualize time-dependent parameters have recently met a growing acceptance. To investigate the technical feasibility of a study dashboard for monitoring the progress of patient recruitment and trial documentation, we set out to develop a propositional prototype in the form of a separate software module.

METHODS

After narrowing down functional requirements in semi-structured interviews with study coordinators, we analyzed available interfaces of a locally deployed EDC application, and designed the prototypical study dashboard based on previous findings. The module thereby leveraged a standardized export format in order to extract and import relevant trial data into a clinical data warehouse. Web-based reporting tools then facilitated the definition of diverse views, including diagrams of the progress of patient accrual and form completion at different granularity levels. To estimate the utility of the dashboard and its compatibility with current workflows, we interviewed study coordinators after a demonstration of sample outputs from ongoing trials.

RESULTS

The employed tools promoted a rapid development. Displays of the implemented dashboard are organized around an entry page that integrates key metrics for available studies, and which links to more detailed information such as study-specific enrollment per center. The interviewed experts commented that the included graphical summaries appeared suitable for detecting that something was generally amiss, although practical remedies would mostly depend on additional information such as access to the original patient-specific data. The dependency on a separate application was seen as a downside. Interestingly, the prospective users warned that in some situations knowledge of specific accrual statistics might undermine blinding in a subtle yet intricate fashion, so ignorance of certain patient features was seen as sometimes preferable for reproducibility.

DISCUSSION

Our proposed study dashboard graphically recaps key progress indicators of patient accrual and trial documentation. The modular implementation illustrates the technical feasibility of the approach. The use of a study dashboard might introduce certain technical requirements as well as subtle interpretative complexities, which may have to be weighed against potential efficiency gains.

Computerized patient identification for the EMBRACA clinical trial using real-time data from the PRAEGNANT network for metastatic breast cancer patients

As breast cancer is a diverse disease, clinical trials are becoming increasingly diversified and are consequently being conducted in very small subgroups of patients, making study recruitment increasingly difficult. The aim of this study was to assess the use of data from a remote data entry system that serves a large national registry for metastatic breast cancer. The PRAEGNANT network is a real-time registry with an integrated biomaterials bank that was designed as a scientific study and as a means of identifying patients who are eligible for clinical trials, based on clinical and molecular information. Here, we report on the automated use of the clinical data documented to identify patients for a clinical trial (EMBRACA) for patients with metastatic breast cancer. The patients' charts were assessed by two independent physicians involved in the clinical trial and also by a computer program that tested patients for eligibility using a structured query language script. In all, 326 patients from two study sites in the PRAEGNANT network were included in the analysis. Using expert assessment, 120 of the 326 patients (37 %) appeared to be eligible for inclusion in the EMBRACA study; with the computer algorithm assessment, a total of 129 appeared to be eligible. The sensitivity of the computer algorithm was 0.87 and its specificity was 0.88. Using computer-based identification of patients for clinical trials appears feasible. With the instrument's high specificity, its application in a large cohort of patients appears to be feasible, and the workload for reassessing the patients is limited.

The INTEGRATE project: Delivering solutions for efficient multi-centric clinical research and trials

The objective of the INTEGRATE project (http://www.fp7-integrate.eu/) that has recently concluded successfully was the development of innovative biomedical applications focused on streamlining the execution of clinical research, on enabling multidisciplinary collaboration, on management and large-scale sharing of multi-level heterogeneous datasets, and on the development of new methodologies and of predictive multi-scale models in cancer. In this paper, we present the way the INTEGRATE consortium has approached important challenges such as the integration of multi-scale biomedical data in the context of post-genomic clinical trials, the development of predictive models and the implementation of tools to facilitate the efficient execution of postgenomic multi-centric clinical trials in breast cancer. Furthermore, we provide a number of key "lessons learned" during the process and give directions for further future research and development.