Synergies and distinctions between computational disciplines in biomedical research: perspective from the Clinical andTranslational Science Award programs

Understanding enterprise data warehouses to support clinical and translational research: enterprise information technology relationships, data governance, workforce, and cloud computing

OBJECTIVE

Among National Institutes of Health Clinical and Translational Science Award (CTSA) hubs, effective approaches for enterprise data warehouses for research (EDW4R) development, maintenance, and sustainability remain unclear. The goal of this qualitative study was to understand CTSA EDW4R operations within the broader contexts of academic medical centers and technology.

MATERIALS AND METHODS

We performed a directed content analysis of transcripts generated from semistructured interviews with informatics leaders from 20 CTSA hubs.

RESULTS

Respondents referred to services provided by health system, university, and medical school information technology (IT) organizations as "enterprise information technology (IT)." Seventy-five percent of respondents stated that the team providing EDW4R service at their hub was separate from enterprise IT; strong relationships between EDW4R teams and enterprise IT were critical for success. Managing challenges of EDW4R staffing was made easier by executive leadership support. Data governance appeared to be a work in progress, as most hubs reported complex and incomplete processes, especially for commercial data sharing. Although nearly all hubs (n = 16) described use of cloud computing for specific projects, only 2 hubs reported using a cloud-based EDW4R. Respondents described EDW4R cloud migration facilitators, barriers, and opportunities.

DISCUSSION

Descriptions of approaches to how EDW4R teams at CTSA hubs work with enterprise IT organizations, manage workforces, make decisions about data, and approach cloud computing provide insights for institutions seeking to leverage patient data for research.

CONCLUSION

Identification of EDW4R best practices is challenging, and this study helps identify a breadth of viable options for CTSA hubs to consider when implementing EDW4R services.

Understanding enterprise data warehouses to support clinical and translational research

OBJECTIVE

Among National Institutes of Health Clinical and Translational Science Award (CTSA) hubs, adoption of electronic data warehouses for research (EDW4R) containing data from electronic health record systems is nearly ubiquitous. Although benefits of EDW4R include more effective, efficient support of scientists, little is known about how CTSA hubs have implemented EDW4R services. The goal of this qualitative study was to understand the ways in which CTSA hubs have operationalized EDW4R to support clinical and translational researchers.

MATERIALS AND METHODS

After conducting semistructured interviews with informatics leaders from 20 CTSA hubs, we performed a directed content analysis of interview notes informed by naturalistic inquiry.

RESULTS

We identified 12 themes: organization and data; oversight and governance; data access request process; data access modalities; data access for users with different skill sets; engagement, communication, and literacy; service management coordinated with enterprise information technology; service management coordinated within a CTSA hub; service management coordinated between informatics and biostatistics; funding approaches; performance metrics; and future trends and current technology challenges.

DISCUSSION

This study is a step in developing an improved understanding and creating a common vocabulary about EDW4R operations across institutions. Findings indicate an opportunity for establishing best practices for EDW4R operations in academic medicine. Such guidance could reduce the costs associated with developing an EDW4R by establishing a clear roadmap and maturity path for institutions to follow.

CONCLUSIONS

CTSA hubs described varying approaches to EDW4R operations that may assist other institutions in better serving investigators with electronic patient data.

Design and virtual implementation of a biomedical registry framework for the enhancement of clinical trials: colorectal cancer example

Introduction

Clinical trials generate a large volume of literature and a vast amount of data. Following the 'open science' model, data sharing has enormous potential to strengthen scientific research. Currently, to the best of our knowledge, there is no existing web based Hellenic biomedical registry that displays available patients for clinical trials, providing direct access to registered physicians to all data, assisting them in finding eligible patients in the initial clinical trial recruitment process.

Methods

This paper describes the design and virtual implementation of a web based prototype biomedical registry in Greece. The system represents an eGovernment framework proposal for the central storage of patients' biomedical information and the operations associated with this process. The increasing tendency to include molecular data as prerequisite elements in clinical trials is adopted in the registry philosophy. The designed system is based on free, open source software and it is implemented virtually on a local host environment.

Results

Using colorectal cancer as an example, valid data from patients increases the reliability index, demonstrating the functionality of the web application.

Conclusion

In conclusion, the combination of biomedical data and information technology in order to display potential participants per health unit, facilitates recruitment for clinical trials.

Research IT maturity models for academic health centers: Early development and initial evaluation

This paper proposes the creation and application of maturity models to guide institutional strategic investment in research informatics and information technology (research IT) and to provide the ability to measure readiness for clinical and research infrastructure as well as sustainability of expertise. Conducting effective and efficient research in health science increasingly relies upon robust research IT systems and capabilities. Academic health centers are increasing investments in health IT systems to address operational pressures, including rapidly growing data, technological advances, and increasing security and regulatory challenges associated with data access requirements. Current approaches for planning and investment in research IT infrastructure vary across institutions and lack comparable guidance for evaluating investments, resulting in inconsistent approaches to research IT implementation across peer academic health centers as well as uncertainty in linking research IT investments to institutional goals. Maturity models address these issues through coupling the assessment of current organizational state with readiness for deployment of potential research IT investment, which can inform leadership strategy. Pilot work in maturity model development has ranged from using them as a catalyst for engaging medical school IT leaders in planning at a single institution to developing initial maturity indices that have been applied and refined across peer medical schools.

Utilizing an integrated infrastructure for outcomes research: a systematic review

OBJECTIVE

To explore the ability of an integrated health information infrastructure to support outcomes research.

METHODS

A systematic review of articles published from 1983 to 2012 by Regenstrief Institute investigators using data from an integrated electronic health record infrastructure involving multiple provider organisations was performed. Articles were independently assessed and classified by study design, disease and other metadata including bibliometrics.

RESULTS

A total of 190 articles were identified. Diseases included cognitive, (16) cardiovascular, (16) infectious, (15) chronic illness (14) and cancer (12). Publications grew steadily (26 in the first decade vs. 100 in the last) as did the number of investigators (from 15 in 1983 to 62 in 2012). The proportion of articles involving non-Regenstrief authors also expanded from 54% in the first decade to 72% in the last decade. During this period, the infrastructure grew from a single health system into a health information exchange network covering more than 6 million patients. Analysis of journal and article metrics reveals high impact for clinical trials and comparative effectiveness research studies that utilised data available in the integrated infrastructure.

DISCUSSION

Integrated information infrastructures support growth in high quality observational studies and diverse collaboration consistent with the goals for the learning health system. More recent publications demonstrate growing external collaborations facilitated by greater access to the infrastructure and improved opportunities to study broader disease and health outcomes.

CONCLUSIONS

Integrated information infrastructures can stimulate learning from electronic data captured during routine clinical care but require time and collaboration to reach full potential.

Neuroinformatics Software Applications Supporting Electronic Data Capture, Management, and Sharing for the Neuroimaging Community

Accelerating insight into the relation between brain and behavior entails conducting small and large-scale research endeavors that lead to reproducible results. Consensus is emerging between funding agencies, publishers, and the research community that data sharing is a fundamental requirement to ensure all such endeavors foster data reuse and fuel reproducible discoveries. Funding agency and publisher mandates to share data are bolstered by a growing number of data sharing efforts that demonstrate how information technologies can enable meaningful data reuse. Neuroinformatics evaluates scientific needs and develops solutions to facilitate the use of data across the cognitive and neurosciences. For example, electronic data capture and management tools designed to facilitate human neurocognitive research can decrease the setup time of studies, improve quality control, and streamline the process of harmonizing, curating, and sharing data across data repositories. In this article we outline the advantages and disadvantages of adopting software applications that support these features by reviewing the tools available and then presenting two contrasting neuroimaging study scenarios in the context of conducting a cross-sectional and a multisite longitudinal study.

Characteristics desired in clinical data warehouse for biomedical research

Objectives

Due to the unique characteristics of clinical data, clinical data warehouses (CDWs) have not been successful so far. Specifically, the use of CDWs for biomedical research has been relatively unsuccessful thus far. The characteristics necessary for the successful implementation and operation of a CDW for biomedical research have not clearly defined yet.

Methods

Three examples of CDWs were reviewed: a multipurpose CDW in a hospital, a CDW for independent multi-institutional research, and a CDW for research use in an institution. After reviewing the three CDW examples, we propose some key characteristics needed in a CDW for biomedical research.

Results

A CDW for research should include an honest broker system and an Institutional Review Board approval interface to comply with governmental regulations. It should also include a simple query interface, an anonymized data review tool, and a data extraction tool. Also, it should be a biomedical research platform for data repository use as well as data analysis.

Conclusions

The proposed characteristics desired in a CDW may have limited transfer value to organizations in other countries. However, these analysis results are still valid in Korea, and we have developed clinical research data warehouse based on these desiderata.

The National Institutes of Health's Biomedical Translational Research Information System (BTRIS): design, contents, functionality and experience to date

The US National Institutes of Health (NIH) has developed the Biomedical Translational Research Information System (BTRIS) to support researchers' access to translational and clinical data. BTRIS includes a data repository, a set of programs for loading data from NIH electronic health records and research data management systems, an ontology for coding the disparate data with a single terminology, and a set of user interface tools that provide access to identified data from individual research studies and data across all studies from which individually identifiable data have been removed. This paper reports on unique design elements of the system, progress to date and user experience after five years of development and operation.

Gap analysis of biomedical informatics graduate education competencies

Graduate training in biomedical informatics (BMI) is evolving rapidly. BMI graduate programs differ in informatics domain, delivery method, degrees granted, as well as breadth and depth of curricular competencies. Using the current American Medical Informatics Association (AMIA) definition of BMI core competencies as a framework, we identified and labeled course offerings within graduate programs. From our qualitative analysis, gaps between defined competencies and curricula emerged. Topics missing from existing graduate curricula include community health, translational and clinical research, knowledge representation, data mining, communication and evidence-based practice.

Recommendations for the use of operational electronic health record data in comparative effectiveness research

There is an increasing amount of clinical data in operational electronic health record (EHR) systems. Such data provide substantial opportunities for their re-use for many purposes, including comparative effectiveness research (CER). In a previous paper, we identified a number of caveats related to the use of such data, noting that they may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, or incompatible with research protocols. In this paper, we provide recommendations for overcoming these caveats with the goal of leveraging such data to benefit CER and other health care activities. These recommendations include adaptation of "best evidence" approaches to use of data; processes to evaluate availability, completeness, quality, and transformability of data; creation of tools to manage data and their attributes; determination of metrics for assessing whether data are "research grade"; development of methods for comparative validation of data; construction of a methodology database for methods involving use of clinical data; standardized reporting methods for data and their attributes; appropriate use of informatics expertise; and a research agenda to determine biases inherent in operational data and to assess informatics approaches to their improvement.

Personalized medicine: challenges and opportunities for translational bioinformatics

Personalized medicine can be defined broadly as a model of healthcare that is predictive, personalized, preventive and participatory. Two US President's Council of Advisors on Science and Technology reports illustrate challenges in personalized medicine (in a 2008 report) and in use of health information technology (in a 2010 report). Translational bioinformatics is a field that can help address these challenges and is defined by the American Medical Informatics Association as "the development of storage, analytic and interpretive methods to optimize the transformation of increasing voluminous biomedical data into proactive, predictive, preventative and participatory health." This article discusses barriers to implementing genomics applications and current progress toward overcoming barriers, describes lessons learned from early experiences of institutions engaged in personalized medicine and provides example areas for translational bioinformatics research inquiry.

People, organizational, and leadership factors impacting informatics support for clinical and translational research

BACKGROUND

In recent years, there have been numerous initiatives undertaken to describe critical information needs related to the collection, management, analysis, and dissemination of data in support of biomedical research (J Investig Med 54:327-333, 2006); (J Am Med Inform Assoc 16:316-327, 2009); (Physiol Genomics 39:131-140, 2009); (J Am Med Inform Assoc 18:354-357, 2011). A common theme spanning such reports has been the importance of understanding and optimizing people, organizational, and leadership factors in order to achieve the promise of efficient and timely research (J Am Med Inform Assoc 15:283-289, 2008). With the emergence of clinical and translational science (CTS) as a national priority in the United States, and the corresponding growth in the scale and scope of CTS research programs, the acuity of such information needs continues to increase (JAMA 289:1278-1287, 2003); (N Engl J Med 353:1621-1623, 2005); (Sci Transl Med 3:90, 2011). At the same time, systematic evaluations of optimal people, organizational, and leadership factors that influence the provision of data, information, and knowledge management technologies and methods are notably lacking.

METHODS

In response to the preceding gap in knowledge, we have conducted both: 1) a structured survey of domain experts at Academic Health Centers (AHCs); and 2) a subsequent thematic analysis of public-domain documentation provided by those same organizations. The results of these approaches were then used to identify critical factors that may influence access to informatics expertise and resources relevant to the CTS domain.

RESULTS

A total of 31 domain experts, spanning the Biomedical Informatics (BMI), Computer Science (CS), Information Science (IS), and Information Technology (IT) disciplines participated in a structured surveyprocess. At a high level, respondents identified notable differences in theaccess to BMI, CS, and IT expertise and services depending on the establishment of a formal BMI academic unit and the perceived relationship between BMI, CS, IS, and IT leaders. Subsequent thematic analysis of the aforementioned public domain documents demonstrated a discordance between perceived and reported integration across and between BMI, CS, IS, and IT programs and leaders with relevance to the CTS domain.

CONCLUSION

Differences in people, organization, and leadership factors do influence the effectiveness of CTS programs, particularly with regard to the ability to access and leverage BMI, CS, IS, and IT expertise and resources. Based on this finding, we believe that the development of a better understanding of how optimal BMI, CS, IS, and IT organizational structures and leadership models are designed and implemented is critical to both the advancement of CTS and ultimately, to improvements in the quality, safety, and effectiveness of healthcare.

Research data collection methods: from paper to tablet computers

BACKGROUND

Primary data collection is a critical activity in clinical research. Even with significant advances in technical capabilities, clear benefits of use, and even user preferences for using electronic systems for collecting primary data, paper-based data collection is still common in clinical research settings. However, with recent developments in both clinical research and tablet computer technology, the comparative advantages and disadvantages of data collection methods should be determined.

OBJECTIVE

To describe case studies using multiple methods of data collection, including next-generation tablets, and consider their various advantages and disadvantages.

MATERIALS AND METHODS

We reviewed 5 modern case studies using primary data collection, using methods ranging from paper to next-generation tablet computers. We performed semistructured telephone interviews with each project, which considered factors relevant to data collection. We address specific issues with workflow, implementation and security for these different methods, and identify differences in implementation that led to different technology considerations for each case study.

RESULTS AND DISCUSSION

There remain multiple methods for primary data collection, each with its own strengths and weaknesses. Two recent methods are electronic health record templates and next-generation tablet computers. Electronic health record templates can link data directly to medical records, but are notably difficult to use. Current tablet computers are substantially different from previous technologies with regard to user familiarity and software cost. The use of cloud-based storage for tablet computers, however, creates a specific challenge for clinical research that must be considered but can be overcome.

AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline

The AMIA biomedical informatics (BMI) core competencies have been designed to support and guide graduate education in BMI, the core scientific discipline underlying the breadth of the field's research, practice, and education. The core definition of BMI adopted by AMIA specifies that BMI is 'the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving and decision making, motivated by efforts to improve human health.' Application areas range from bioinformatics to clinical and public health informatics and span the spectrum from the molecular to population levels of health and biomedicine. The shared core informatics competencies of BMI draw on the practical experience of many specific informatics sub-disciplines. The AMIA BMI analysis highlights the central shared set of competencies that should guide curriculum design and that graduate students should be expected to master.

Probabilistic techniques for obtaining accurate patient counts in Clinical Data Warehouses

Proposal and execution of clinical trials, computation of quality measures and discovery of correlation between medical phenomena are all applications where an accurate count of patients is needed. However, existing sources of this type of patient information, including Clinical Data Warehouses (CDWs) may be incomplete or inaccurate. This research explores applying probabilistic techniques, supported by the MayBMS probabilistic database, to obtain accurate patient counts from a Clinical Data Warehouse containing synthetic patient data. We present a synthetic Clinical Data Warehouse, and populate it with simulated data using a custom patient data generation engine. We then implement, evaluate and compare different techniques for obtaining patients counts. We model billing as a test for the presence of a condition. We compute billing's sensitivity and specificity both by conducting a "Simulated Expert Review" where a representative sample of records are reviewed and labeled by experts, and by obtaining the ground truth for every record. We compute the posterior probability of a patient having a condition through a "Bayesian Chain", using Bayes' Theorem to calculate the probability of a patient having a condition after each visit. The second method is a "one-shot" approach that computes the probability of a patient having a condition based on whether the patient is ever billed for the condition. Our results demonstrate the utility of probabilistic approaches, which improve on the accuracy of raw counts. In particular, the simulated review paired with a single application of Bayes' Theorem produces the best results, with an average error rate of 2.1% compared to 43.7% for the straightforward billing counts. Overall, this research demonstrates that Bayesian probabilistic approaches improve patient counts on simulated patient populations. We believe that total patient counts based on billing data are one of the many possible applications of our Bayesian framework. Use of these probabilistic techniques will enable more accurate patient counts and better results for applications requiring this metric.

DW4TR: A Data Warehouse for Translational Research

The linkage between the clinical and laboratory research domains is a key issue in translational research. Integration of clinicopathologic data alone is a major task given the number of data elements involved. For a translational research environment, it is critical to make these data usable at the point-of-need. Individual systems have been developed to meet the needs of particular projects though the need for a generalizable system has been recognized. Increased use of Electronic Medical Record data in translational research will demand generalizing the system for integrating clinical data to support the study of a broad range of human diseases. To ultimately satisfy these needs, we have developed a system to support multiple translational research projects. This system, the Data Warehouse for Translational Research (DW4TR), is based on a light-weight, patient-centric modularly-structured clinical data model and a specimen-centric molecular data model. The temporal relationships of the data are also part of the model. The data are accessed through an interface composed of an Aggregated Biomedical-Information Browser (ABB) and an Individual Subject Information Viewer (ISIV) which target general users. The system was developed to support a breast cancer translational research program and has been extended to support a gynecological disease program. Further extensions of the DW4TR are underway. We believe that the DW4TR will play an important role in translational research across multiple disease types.

Biomedical informatics: changing what physicians need to know and how they learn

The explosive growth of biomedical complexity calls for a shift in the paradigm of medical decision making-from a focus on the power of an individual brain to the collective power of systems of brains. This shift alters professional roles and requires biomedical informatics and information technology (IT) infrastructure. The authors illustrate this future role of medical informatics with a vignette and summarize the evolving understanding of both beneficial and deleterious effects of informatics-rich environments on learning, clinical care, and research. The authors also provide a framework of core informatics competencies for health professionals of the future and conclude with broad steps for faculty development. They recommend that medical schools advance on four fronts to prepare their faculty to teach in a biomedical informatics-rich world: (1) create academic units in biomedical informatics; (2) adapt the IT infrastructure of academic health centers (AHCs) into testing laboratories; (3) introduce medical educators to biomedical informatics sufficiently for them to model its use; and (4) retrain AHC faculty to lead the transformation to health care based on a new systems approach enabled by biomedical informatics. The authors propose that embracing this collective and informatics-enhanced future of medicine will provide opportunities to advance education, patient care, and biomedical science.

Characterizing Data Discovery and End-User Computing Needs in Clinical Translational Science

In this paper, the authors present the results of a qualitative case-study seeking to characterize data discovery needs and barriers of principal investigators and research support staff in clinical translational science. Several implications for designing and implementing translational research systems have emerged through the authors' analysis. The results also illustrate the benefits of forming early partnerships with scientists to better understand their workflow processes and end-user computing practices in accessing data for research. The authors use this user-centered, iterative development approach to guide the implementation and extension of i2b2, a system they have adapted to support cross-institutional aggregate anonymized clinical data querying. With ongoing evaluation, the goal is to maximize the utility and extension of this system and develop an interface that appropriately fits the swiftly evolving needs of clinical translational scientists.

Linking scientific discovery and better health for the nation: the first three years of the NIH's Clinical and Translational Science Awards

A comprehensive system for translating basic biomedical research into useful and effectively implemented clinical diagnostic, preventive, and therapeutic practices is essential to the nation's health. The state of clinical and translational research (CTR) in the United States, however, has been characterized as fragmented, slow, expensive, and poorly coordinated. As part of its Roadmap Initiative, the National Institutes of Health instituted the Clinical and Translational Science Awards (CTSA), a sweeping and ambitious program designed to transform the conduct of biomedical research in the United States by speeding the translation of scientific discoveries into useful therapies and then developing methods to ensure that those therapies reach the patients who need them the most. The authors review the circumstances of the U.S. biomedical research enterprise that led to the creation of the CTSA and discuss the initial strategic plan of the CTSA, which was developed from the first three years of experience with the program and was designed to overcome organizational, methodological, and cultural barriers within and among research institutions. The authors also describe the challenges encountered during these efforts and discuss the promise of this vital national health care initiative, which is essential to creating a pipeline for the scientific workforce needed to conduct research that will, in turn, provide a rational evidence base for better health in the United States.

Biomedical informatics and translational medicine

Biomedical informatics involves a core set of methodologies that can provide a foundation for crossing the "translational barriers" associated with translational medicine. To this end, the fundamental aspects of biomedical informatics (e.g., bioinformatics, imaging informatics, clinical informatics, and public health informatics) may be essential in helping improve the ability to bring basic research findings to the bedside, evaluate the efficacy of interventions across communities, and enable the assessment of the eventual impact of translational medicine innovations on health policies. Here, a brief description is provided for a selection of key biomedical informatics topics (Decision Support, Natural Language Processing, Standards, Information Retrieval, and Electronic Health Records) and their relevance to translational medicine. Based on contributions and advancements in each of these topic areas, the article proposes that biomedical informatics practitioners ("biomedical informaticians") can be essential members of translational medicine teams.

Predictors of student success in graduate biomedical informatics training: introductory course and program success

OBJECTIVE

To predict student performance in an introductory graduate-level biomedical informatics course from application data.

DESIGN

A predictive model built through retrospective review of student records using hierarchical binary logistic regression with half of the sample held back for cross-validation. The model was also validated against student data from a similar course at a second institution.

MEASUREMENTS

Earning an A grade (Mastery) or a C grade (Failure) in an introductory informatics course.

RESULTS

The authors analyzed 129 student records at the University of Texas School of Health Information Sciences at Houston (SHIS) and 106 at Oregon Health and Science University Department of Medical Informatics and Clinical Epidemiology (DMICE). In the SHIS cross-validation sample, the Graduate Record Exam verbal score (GRE-V) correctly predicted Mastery in 69.4%. Undergraduate grade point average (UGPA) and underrepresented minority status (URMS) predicted 81.6% of Failures. At DMICE, GRE-V, UGPA, and prior graduate degree significantly correlated with Mastery. Only GRE-V was a significant independent predictor of Mastery at both institutions. There were too few URMS students and Failures at DMICE to analyze. Course Mastery strongly predicted program performance defined as final cumulative GPA at SHIS (n=19, r=0.634, r2=0.40, p=0.0036) and DMICE (n=106, r=0.603, r2=0.36, p<0.001).

CONCLUSIONS

The authors identified predictors of performance in an introductory informatics course including GRE-V, UGPA and URMS. Course performance was a very strong predictor of overall program performance. Findings may be useful for selecting students for admission and identifying students at risk for Failure as early as possible.

A stimulus to define informatics and health information technology

BACKGROUND

Despite the growing interest by leaders, policy makers, and others, the terminology of health information technology as well as biomedical and health informatics is poorly understood and not even agreed upon by academics and professionals in the field.

DISCUSSION

The paper, presented as a Debate to encourage further discussion and disagreement, provides definitions of the major terminology used in biomedical and health informatics and health information technology. For informatics, it focuses on the words that modify the term as well as individuals who practice the discipline. Other categories of related terms are covered as well, from the associated disciplines of computer science, information technology and health information management to the major application categories of applications used. The discussion closes with a classification of individuals who work in the largest segment of the field, namely clinical informatics.

SUMMARY

The goal of presenting in Debate format is to provide a starting point for discussion to reach a documented consensus on the definition and use of these terms.