People and organizational issues in research systems implementation

Electronic Data Capture System (REDCap) for Health Care Research and Training in a Resource-Constrained Environment: Technology Adoption Case Study

Background

Electronic data capture (EDC) in academic health care organizations provides an opportunity for the management, aggregation, and secondary use of research and clinical data. It is especially important in resource-constrained environments such as the South African public health care sector, where paper records are still the main form of clinical record keeping.

Objective

The aim of this study was to describe the strategies followed by the University of the Witwatersrand Faculty of Health Sciences (Wits FHS) during the period from 2013 to 2021 to overcome resistance to, and encourage the adoption of, the REDCap (Research Electronic Data Capture; Vanderbilt University) system by academic and clinical staff. REDCap has found wide use in varying domains, including clinical studies and research projects as well as administrative, financial, and human resource applications. Given REDCap’s global footprint in >5000 institutions worldwide and potential for future growth, the strategies followed by the Wits FHS to support users and encourage adoption may be of importance to others using the system, particularly in resource-constrained settings.

Methods

The strategies to support users and encourage adoption included top-down organizational support; secure and reliable application, hosting infrastructure, and systems administration; an enabling and accessible REDCap support team; regular hands-on training workshops covering REDCap project setup and data collection instrument design techniques; annual local symposia to promote networking and awareness of all the latest software features and best practices for using them; participation in REDCap Consortium activities; and regular and ongoing mentorship from members of the Vanderbilt University Medical Center.

Results

During the period from 2013 to 2021, the use of the REDCap EDC system by individuals at the Wits FHS increased, respectively, from 129 active user accounts to 3447 active user accounts. The number of REDCap projects increased from 149 in 2013 to 12,865 in 2021. REDCap at Wits also supported various publications and research outputs, including journal articles and postgraduate monographs. As of 2020, a total of 233 journal articles and 87 postgraduate monographs acknowledged the use of the Wits REDCap system.

Conclusions

By providing reliable infrastructure and accessible support resources, we were able to successfully implement and grow the REDCap EDC system at the Wits FHS and its associated academic medical centers. We believe that the increase in the use of REDCap was driven by offering a dependable, secure service with a strong end-user training and support model. This model may be applied by other academic and health care organizations in resource-constrained environments planning to implement EDC technology.

Leveraging Real-World Data for the Selection of Relevant Eligibility Criteria for the Implementation of Electronic Recruitment Support in Clinical Trials

BACKGROUND

Even though clinical trials are indispensable for medical research, they are frequently impaired by delayed or incomplete patient recruitment, resulting in cost overruns or aborted studies. Study protocols based on real-world data with precisely expressed eligibility criteria and realistic cohort estimations are crucial for successful study execution. The increasing availability of routine clinical data in electronic health records (EHRs) provides the opportunity to also support patient recruitment during the prescreening phase. While solutions for electronic recruitment support have been published, to our knowledge, no method for the prioritization of eligibility criteria in this context has been explored.

METHODS

In the context of the Electronic Health Records for Clinical Research (EHR4CR) project, we examined the eligibility criteria of the KATHERINE trial. Criteria were extracted from the study protocol, deduplicated, and decomposed. A paper chart review and data warehouse query were executed to retrieve clinical data for the resulting set of simplified criteria separately from both sources. Criteria were scored according to disease specificity, data availability, and discriminatory power based on their content and the clinical dataset.

RESULTS

The study protocol contained 35 eligibility criteria, which after simplification yielded 70 atomic criteria. For a cohort of 106 patients with breast cancer and neoadjuvant treatment, 47.9% of data elements were captured through paper chart review, with the data warehouse query yielding 26.9% of data elements. Score application resulted in a prioritized subset of 17 criteria, which yielded a sensitivity of 1.00 and specificity 0.57 on EHR data (paper charts, 1.00 and 0.80) compared with actual recruitment in the trial.

CONCLUSION

It is possible to prioritize clinical trial eligibility criteria based on real-world data to optimize prescreening of patients on a selected subset of relevant and available criteria and reduce implementation efforts for recruitment support. The performance could be further improved by increasing EHR data coverage.

Institutional ELN/LIMS deployment: Highly customizable ELN/LIMS platform as a cornerstone of digital transformation for life sciences research institutes

The systematic recording and management of experimental data in academic life science research remains an open problem. EPFL engaged in a program of ELN/LIMS deployment 6 years ago, encountering a host of fundamental questions at the institutional level and in each single laboratory. Here, based on our experience we aim to share with research institute managers, PIs, and any scientists involved in an ELN/LIMS deployment, helpful tips and tools to surround yourself with the right people and the right software at the right time. In this article we describe the resources used, the challenges, key success factors, and the results obtained at each phase of our project. Finally, we discuss the current and next challenges and how our experience leads us to support the creation of a new position in the research groups: the laboratory data manager.

The use of an electronic health record system reduces errors in the National Hip Fracture Database

AIM

to compare the validity of data submitted from a UK level 1 trauma centre to the National Hip Fracture Database (NHFD) before and after the introduction of an electronic health record system (EHRS).

PATIENTS AND METHODS

a total of 3224 records were reviewed from July 2009 to July 2017. 2,133 were submitted between July 2009 and October 2014 and 1,091 between October 2014 and July 2017, representing data submitted before and after the introduction of the EHRS, respectively. Data submitted to the NHFD were scrutinised against locally held data.

RESULTS

use of an EHRS was associated with significant reductions in NHFD errors. The operation coding error rate fell significantly from 23.2% (494/2133) to 7.6% (83/1091); P < 0.001. Prior to EHRS introduction, of the 109 deaths recorded in the NHFD, 64 (59%) were incorrect. In the EHRS dataset, all the 112 recorded deaths were correct (P < 0.001). There was no significant difference in the error rate for fracture coding. In the EHRS dataset, after controlling for sample month, entries utilising an operation note template with mandatory fields relevant to NHFD data were more likely to be error free than those not using the template (OR 2.69; 95% CI 1.92-3.78).

CONCLUSION

this study highlights a potential benefit of EHR systems, which offer automated data collection for auditing purposes. However, errors in data submitted to the NHFD remain, particularly in cases where an NHFD-specific operation note template is not used. Clinician engagement with new technologies is vital to avoid human error and ensure database integrity.

Challenges in patient safety improvement research in the era of electronic health records

Electronic health record (EHR) data repositories contain large volumes of aggregated, longitudinal clinical data that could allow patient safety researchers to identify important safety issues and conduct comprehensive evaluations of health care delivery outcomes. However, few health systems have successfully converted this abundance of data into useful information or knowledge for safety improvement. In this paper, we use a case study involving a project on missed/delayed follow-up of test results to discuss real-world challenges in using EHR data for patient safety research. We identify three types of challenges that pose as barriers to advance patient safety improvement research: 1) gaining approval to access/review EHR data; 2) interpreting EHR data; 3) working with local IT/EHR personnel. We discuss the complexity of these challenges, all of which are unlikely to be unique to this project, and outline some key next steps that must be taken to support research that uses EHR data to improve safety. We recognize that all organizations face competing priorities between clinical operations and research. However, to leverage EHRs and their abundant data for patient safety improvement research, many current data access and security policies and procedures must be rewritten and standardized across health care organizations. These efforts are essential to help make EHRs and EHR data useful for progress in our journey to safer health care.

Implementing partnership-driven clinical federated electronic health record data sharing networks

OBJECTIVE

Building federated data sharing architectures requires supporting a range of data owners, effective and validated semantic alignment between data resources, and consistent focus on end-users. Establishing these resources requires development methodologies that support internal validation of data extraction and translation processes, sustaining meaningful partnerships, and delivering clear and measurable system utility. We describe findings from two federated data sharing case examples that detail critical factors, shared outcomes, and production environment results.

METHODS

Two federated data sharing pilot architectures developed to support network-based research associated with the University of Washington's Institute of Translational Health Sciences provided the basis for the findings. A spiral model for implementation and evaluation was used to structure iterations of development and support knowledge share between the two network development teams, which cross collaborated to support and manage common stages.

RESULTS

We found that using a spiral model of software development and multiple cycles of iteration was effective in achieving early network design goals. Both networks required time and resource intensive efforts to establish a trusted environment to create the data sharing architectures. Both networks were challenged by the need for adaptive use cases to define and test utility.

CONCLUSION

An iterative cyclical model of development provided a process for developing trust with data partners and refining the design, and supported measureable success in the development of new federated data sharing architectures.

Resolving complex research data management issues in biomedical laboratories: Qualitative study of an industry-academia collaboration

This paper describes a distributed collaborative effort between industry and academia to systematize data management in an academic biomedical laboratory. Heterogeneous and voluminous nature of research data created in biomedical laboratories make information management difficult and research unproductive. One such collaborative effort was evaluated over a period of four years using data collection methods including ethnographic observations, semi-structured interviews, web-based surveys, progress reports, conference call summaries, and face-to-face group discussions. Data were analyzed using qualitative methods of data analysis to (1) characterize specific problems faced by biomedical researchers with traditional information management practices, (2) identify intervention areas to introduce a new research information management system called Labmatrix, and finally to (3) evaluate and delineate important general collaboration (intervention) characteristics that can optimize outcomes of an implementation process in biomedical laboratories. Results emphasize the importance of end user perseverance, human-centric interoperability evaluation, and demonstration of return on investment of effort and time of laboratory members and industry personnel for success of implementation process. In addition, there is an intrinsic learning component associated with the implementation process of an information management system. Technology transfer experience in a complex environment such as the biomedical laboratory can be eased with use of information systems that support human and cognitive interoperability. Such informatics features can also contribute to successful collaboration and hopefully to scientific productivity.

The challenge of benchmarking health systems: is ICT innovation capacity more systemic than organizational dependent?

The article by Catan et al. presents a benchmarking exercise comparing Israel and Portugal on the implementation of Information and Communication Technologies in the healthcare sector. Special attention was given to e-Health and m-Health. The authors collected information via a set of interviews with key stakeholders. They compared two different cultures and societies, which have reached slightly different implementation outcomes. Although the comparison is very enlightening, it is also challenging. Benchmarking exercises present a set of challenges, such as the choice of methodologies and the assessment of the impact on organizational strategy. Precise benchmarking methodology is a valid tool for eliciting information about alternatives for improving health systems. However, many beneficial interventions, which benchmark as effective, fail to translate into meaningful healthcare outcomes across contexts. There is a relationship between results and the innovational and competitive environments. Differences in healthcare governance and financing models are well known; but little is known about their impact on Information and Communication Technology implementation. The article by Catan et al. provides interesting clues about this issue. Public systems (such as those of Portugal, UK, Sweden, Spain, etc.) present specific advantages and disadvantages concerning Information and Communication Technology development and implementation. Meanwhile, private systems based fundamentally on insurance packages, (such as Israel, Germany, Netherlands or USA) present a different set of advantages and disadvantages - especially a more open context for innovation. Challenging issues from both the Portuguese and Israeli cases will be addressed. Clearly, more research is needed on both benchmarking methodologies and on ICT implementation strategies.

Health information exchange implementation: lessons learned and critical success factors from a case study

BACKGROUND

Much attention has been given to the proposition that the exchange of health information as an act, and health information exchange (HIE), as an entity, are critical components of a framework for health care change, yet little has been studied to understand the value proposition of implementing HIE with a statewide HIE. Such an organization facilitates the exchange of health information across disparate systems, thus following patients as they move across different care settings and encounters, whether or not they share an organizational affiliation. A sociotechnical systems approach and an interorganizational systems framework were used to examine implementation of a health system electronic medical record (EMR) system onto a statewide HIE, under a cooperative agreement with the Office of the National Coordinator for Health Information Technology, and its collaborating organizations.

OBJECTIVE

The objective of the study was to focus on the implementation of a health system onto a statewide HIE; provide insight into the technical, organizational, and governance aspects of a large private health system and the Virginia statewide HIE (organizations with the shared goal of exchanging health information); and to understand the organizational motivations and value propositions apparent during HIE implementation.

METHODS

We used a formative evaluation methodology to investigate the first implementation of a health system onto the statewide HIE. Qualitative methods (direct observation, 36 hours), informal information gathering, semistructured interviews (N=12), and document analysis were used to gather data between August 12, 2012 and June 24, 2013. Derived from sociotechnical concepts, a Blended Value Collaboration Enactment Framework guided the data gathering and analysis to understand organizational stakeholders' perspectives across technical, organizational, and governance dimensions.

RESULTS

Several challenges, successes, and lessons learned during the implementation of a health system to the statewide HIE were found. The most significant perceived success was accomplishing the implementation, although many interviewees also underscored the value of a project champion with decision-making power. In terms of lessons learned, social reasons were found to be very significant motivators for early implementation, frequently outweighing economic motivations. It was clear that understanding the guides early in the project would have mitigated some of the challenges that emerged, and early communication with the electronic health record vendor so that they have a solid understanding of the undertaking was critical. An HIE implementations evaluation framework was found to be useful for assessing challenges, motivations, value propositions for participating, and success factors to consider for future implementations.

CONCLUSIONS

This case study illuminates five critical success factors for implementation of a health system onto a statewide HIE. This study also reveals that organizations have varied motivations and value proposition perceptions for engaging in the exchange of health information, few of which, at the early stages, are economically driven.

Implementing a Clinical Research Management System: One Institution's Successful Approach Following Previous Failures

Clinical research management systems (CRMSs) can facilitate research billing compliance and clinician awareness of study activities when integrated with practice management and electronic health record systems. However, adoption of CRMSs remains low, and optimal approaches to implementation are unknown. This case report describes one institution's successful approach to organization, technology, and workflow for CRMS implementation following previous failures. Critical factors for CRMS success included organizational commitment to clinical research, a dedicated research information technology unit, integration of research data across disparate systems, and centralized system usage workflows. In contrast, previous failed approaches at the institution lacked a mandate and mechanism for change, received support as a business rather than research activity, maintained data in separate systems, and relied on inconsistent distributed system usage workflows. To our knowledge, this case report is the first to describe CRMS implementation success and failures, which can assist practitioners and academic evaluators.

Implementing unique device identification in electronic health record systems: organizational, workflow, and technological challenges

BACKGROUND

The United States Food and Drug Administration (FDA) has proposed creating a unique device identification (UDI) system for medical devices to facilitate postmarket surveillance, quality improvement, and other applications. Although a small number of health care institutions have implemented initiatives comparable with the proposed UDI system by capturing data in electronic health record (EHR) systems, it is unknown whether institutions with fewer resources will be able to similarly implement UDI.

OBJECTIVE AND METHODS

This paper calls attention to organizational, workflow, and technological challenges in UDI system implementation by drawing from the literature on EHR and clinical research systems implementation.

FINDINGS

Organizational challenges for UDI system implementation include coordinating multiple stakeholders to define UDI attributes and characteristics for use in EHRs, guiding organizational change within individual institutions for integrating UDI with EHRs, and guiding organizational change for reusing UDI data captured in EHRs. Workflow challenges include capturing UDI data in EHRs using keyboard entry and barcode scanning. Technological challenges involve interfacing UDI data between EHRs and surgical information systems, transforming UDI and related patient data from EHRs for research, and applying data standards to UDI within and beyond EHRs.

DISCUSSION AND CONCLUSIONS

We provide recommendations for regulations, organizational sharing, and professional society engagement to raise awareness of and overcome UDI system implementation challenges. Implementation of the UDI system will require integration of people, process, and technology to achieve benefits envisioned by FDA, including improved postmarket device surveillance and quality of care.

Evaluation of electronic health record implementation in ophthalmology at an academic medical center (an American Ophthalmological Society thesis)

PURPOSE

To evaluate three measures related to electronic health record (EHR) implementation: clinical volume, time requirements, and nature of clinical documentation. Comparison is made to baseline paper documentation.

METHODS

An academic ophthalmology department implemented an EHR in 2006. A study population was defined of faculty providers who worked the 5 months before and after implementation. Clinical volumes, as well as time length for each patient encounter, were collected from the EHR reporting system. To directly compare time requirements, two faculty providers who utilized both paper and EHR systems completed time-motion logs to record the number of patients, clinic time, and nonclinic time to complete documentation. Faculty providers and databases were queried to identify patient records containing both paper and EHR notes, from which three cases were identified to illustrate representative documentation differences.

RESULTS

Twenty-three faculty providers completed 120,490 clinical encounters during a 3-year study period. Compared to baseline clinical volume from 3 months pre-implementation, the post-implementation volume was 88% in quarter 1, 93% in year 1, 97% in year 2, and 97% in year 3. Among all encounters, 75% were completed within 1.7 days after beginning documentation. The mean total time per patient was 6.8 minutes longer with EHR than paper (P<.01). EHR documentation involved greater reliance on textual interpretation of clinical findings, whereas paper notes used more graphical representations, and EHR notes were longer and included automatically generated text.

CONCLUSION

This EHR implementation was associated with increased documentation time, little or no increase in clinical volume, and changes in the nature of ophthalmic documentation.

Advancing user experience research to facilitate and enable patient-centered research: current state and future directions

Human-computer interaction and related areas of user experience (UX) research, such as human factors, workflow evaluation, and data visualization, are thus essential to presenting data in ways that can further the analysis of complex data sets such as those used in patient-centered research. However, a review of available data on the state of UX research as it relates to patient-centered research demonstrates a significant underinvestment and consequently a large gap in knowledge generation. In response, this report explores trends in funding and research productivity focused on UX and patient-centered research and then presents a set of recommendations to advance innovation at this important intersection point. Ultimately, the aim is to catalyze a community-wide dialogue concerning future directions for research and innovation in UX as it applies to patient-centered research.

People, organizational, and leadership factors impacting informatics support for clinical and translational research

BACKGROUND

In recent years, there have been numerous initiatives undertaken to describe critical information needs related to the collection, management, analysis, and dissemination of data in support of biomedical research (J Investig Med 54:327-333, 2006); (J Am Med Inform Assoc 16:316-327, 2009); (Physiol Genomics 39:131-140, 2009); (J Am Med Inform Assoc 18:354-357, 2011). A common theme spanning such reports has been the importance of understanding and optimizing people, organizational, and leadership factors in order to achieve the promise of efficient and timely research (J Am Med Inform Assoc 15:283-289, 2008). With the emergence of clinical and translational science (CTS) as a national priority in the United States, and the corresponding growth in the scale and scope of CTS research programs, the acuity of such information needs continues to increase (JAMA 289:1278-1287, 2003); (N Engl J Med 353:1621-1623, 2005); (Sci Transl Med 3:90, 2011). At the same time, systematic evaluations of optimal people, organizational, and leadership factors that influence the provision of data, information, and knowledge management technologies and methods are notably lacking.

METHODS

In response to the preceding gap in knowledge, we have conducted both: 1) a structured survey of domain experts at Academic Health Centers (AHCs); and 2) a subsequent thematic analysis of public-domain documentation provided by those same organizations. The results of these approaches were then used to identify critical factors that may influence access to informatics expertise and resources relevant to the CTS domain.

RESULTS

A total of 31 domain experts, spanning the Biomedical Informatics (BMI), Computer Science (CS), Information Science (IS), and Information Technology (IT) disciplines participated in a structured surveyprocess. At a high level, respondents identified notable differences in theaccess to BMI, CS, and IT expertise and services depending on the establishment of a formal BMI academic unit and the perceived relationship between BMI, CS, IS, and IT leaders. Subsequent thematic analysis of the aforementioned public domain documents demonstrated a discordance between perceived and reported integration across and between BMI, CS, IS, and IT programs and leaders with relevance to the CTS domain.

CONCLUSION

Differences in people, organization, and leadership factors do influence the effectiveness of CTS programs, particularly with regard to the ability to access and leverage BMI, CS, IS, and IT expertise and resources. Based on this finding, we believe that the development of a better understanding of how optimal BMI, CS, IS, and IT organizational structures and leadership models are designed and implemented is critical to both the advancement of CTS and ultimately, to improvements in the quality, safety, and effectiveness of healthcare.

Chapter 1: Biomedical knowledge integration

The modern biomedical research and healthcare delivery domains have seen an unparalleled increase in the rate of innovation and novel technologies over the past several decades. Catalyzed by paradigm-shifting public and private programs focusing upon the formation and delivery of genomic and personalized medicine, the need for high-throughput and integrative approaches to the collection, management, and analysis of heterogeneous data sets has become imperative. This need is particularly pressing in the translational bioinformatics domain, where many fundamental research questions require the integration of large scale, multi-dimensional clinical phenotype and bio-molecular data sets. Modern biomedical informatics theory and practice has demonstrated the distinct benefits associated with the use of knowledge-based systems in such contexts. A knowledge-based system can be defined as an intelligent agent that employs a computationally tractable knowledge base or repository in order to reason upon data in a targeted domain and reproduce expert performance relative to such reasoning operations. The ultimate goal of the design and use of such agents is to increase the reproducibility, scalability, and accessibility of complex reasoning tasks. Examples of the application of knowledge-based systems in biomedicine span a broad spectrum, from the execution of clinical decision support, to epidemiologic surveillance of public data sets for the purposes of detecting emerging infectious diseases, to the discovery of novel hypotheses in large-scale research data sets. In this chapter, we will review the basic theoretical frameworks that define core knowledge types and reasoning operations with particular emphasis on the applicability of such conceptual models within the biomedical domain, and then go on to introduce a number of prototypical data integration requirements and patterns relevant to the conduct of translational bioinformatics that can be addressed via the design and use of knowledge-based systems.

Auditing medical records accesses via healthcare interaction networks

Healthcare organizations are deploying increasingly complex clinical information systems to support patient care. Traditional information security practices (e.g., role-based access control) are embedded in enterprise-level systems, but are insufficient to ensure patient privacy. This is due, in part, to the dynamic nature of healthcare, which makes it difficult to predict which care providers need access to what and when. In this paper, we show that modeling operations at a higher level of granularity (e.g., the departmental level) are stable in the context of a relational network, which may enable more effective auditing strategies. We study three months of access logs from a large academic medical center to illustrate that departmental interaction networks exhibit certain invariants, such as the number, strength, and reciprocity of relationships. We further show that the relations extracted from the network can be leveraged to assess the extent to which a patient's care satisfies expected organizational behavior.

Lessons learned from implementation of a perinatal documentation system

Implementation of an electronic medical record (EMR) system is a complex process with broad implications. In a Midwestern hospital perinatal setting, EMR implementation involved several critical steps: strategic planning and project goal setting; project structure and governance; system requirements analysis; vendor selection and contract negotiation; and EMR training. No difference in patient care activities and communications among clinicians was found between pre- and post-EMR implementation; however, nurses' perceptions of EMR were more negative afterward. Lessons learned and implications are provided.

Modeling nurses' acceptance of bar coded medication administration technology at a pediatric hospital

OBJECTIVE

To identify predictors of nurses' acceptance of bar coded medication administration (BCMA).

DESIGN

Cross-sectional survey of registered nurses (N=83) at an academic pediatric hospital that recently implemented BCMA.

METHODS

Surveys assessed seven BCMA-related perceptions: ease of use; usefulness for the job; social influence from non-specific others to use BCMA; training; technical support; usefulness for patient care; and social influence from patients/families. An all possible subset regression procedure with five goodness-of-fit indicators was used to identify which set of perceptions best predicted BCMA acceptance (intention to use, satisfaction).

RESULTS

Nurses reported a moderate perceived ease of use and low perceived usefulness of BCMA. Nurses perceived moderate-or-higher social influence to use BCMA and had moderately positive perceptions of BCMA-related training and technical support. Behavioral intention to use BCMA was high, but satisfaction was low. Behavioral intention to use was best predicted by perceived ease of use, perceived social influence from non-specific others, and perceived usefulness for patient care (56% of variance explained). Satisfaction was best predicted by perceived ease of use, perceived usefulness for patient care, and perceived social influence from patients/families (76% of variance explained).

DISCUSSION

Variation in and low scores on ease of use and usefulness are concerning, especially as these variables often correlate with acceptance, as found in this study. Predicting acceptance benefited from using a broad set of perceptions and adapting variables to the healthcare context.

CONCLUSION

Success with BCMA and other technologies can benefit from assessing end-user acceptance and elucidating the factors promoting acceptance and use.

Implementation of a deidentified federated data network for population-based cohort discovery

OBJECTIVE

The Cross-Institutional Clinical Translational Research project explored a federated query tool and looked at how this tool can facilitate clinical trial cohort discovery by managing access to aggregate patient data located within unaffiliated academic medical centers.

METHODS

The project adapted software from the Informatics for Integrating Biology and the Bedside (i2b2) program to connect three Clinical Translational Research Award sites: University of Washington, Seattle, University of California, Davis, and University of California, San Francisco. The project developed an iterative spiral software development model to support the implementation and coordination of this multisite data resource.

RESULTS

By standardizing technical infrastructures, policies, and semantics, the project enabled federated querying of deidentified clinical datasets stored in separate institutional environments and identified barriers to engaging users for measuring utility.

DISCUSSION

The authors discuss the iterative development and evaluation phases of the project and highlight the challenges identified and the lessons learned.

CONCLUSION

The common system architecture and translational processes provide high-level (aggregate) deidentified access to a large patient population (>5 million patients), and represent a novel and extensible resource. Enhancing the network for more focused disease areas will require research-driven partnerships represented across all partner sites.

High-content screening data management for drug discovery in a small- to medium-size laboratory: results of a collaborative pilot study focused on user expectations as indicators of effectiveness

High-content screening (HCS) technology provides a powerful vantage point to approach biological problems; it allows analysis of cell parameters, including changes in cell or protein movement, shape, or texture. As part of a collaborative pilot research project to improve bioscience research data integration, we identified HCS data management as an area ripe for advancement. A primary goal was to develop an integrated data management and analysis system suitable for small- to medium-size HCS programs that would improve research productivity and increase work satisfaction. A system was developed that uses Labmatrix, a Web-based research data management platform, to integrate and query data derived from a Cellomics STORE database. Focusing on user expectations, several barriers to HCS productivity were identified and reduced or eliminated. The impact of the project on HCS research productivity was tested through a series of 18 lab-requested integrated data queries, 7 of which were fully enabled, 7 partially enabled, and 4 enabled through data export to standalone data analysis tools. The results are limited to one laboratory, but this pilot suggests that through an "implementation research" approach, a network of small- to medium-size laboratories involved in HCS projects could achieve greater productivity and satisfaction in drug discovery research.

Assessing the motivation of MDs to use computer-based support at the point-of-care in the Emergency Department

A significant body of research investigates the acceptance of computer-based support (including devices and applications ranging from e-mail to specialized clinical systems, like PACS) among clinicians. Much of this research has focused on measuring the usability of systems using characteristics related to the clarity of interactions and ease of use. We propose that an important attribute of any clinical computer-based support tool is the intrinsic motivation of the end-user (i.e. a clinician) to use the system in practice. In this paper we present the results of a study that investigated factors motivating medical doctors (MDs) to use computer-based support. Our results demonstrate that MDs value computer-based support, find it useful and easy to use, however, uptake is hindered by perceived incompetence, and pressure and tension associated with using technology.

Characterizing Data Discovery and End-User Computing Needs in Clinical Translational Science

In this paper, the authors present the results of a qualitative case-study seeking to characterize data discovery needs and barriers of principal investigators and research support staff in clinical translational science. Several implications for designing and implementing translational research systems have emerged through the authors' analysis. The results also illustrate the benefits of forming early partnerships with scientists to better understand their workflow processes and end-user computing practices in accessing data for research. The authors use this user-centered, iterative development approach to guide the implementation and extension of i2b2, a system they have adapted to support cross-institutional aggregate anonymized clinical data querying. With ongoing evaluation, the goal is to maximize the utility and extension of this system and develop an interface that appropriately fits the swiftly evolving needs of clinical translational scientists.

Organization of Biomedical Data for Collaborative Scientific Research: A Research Information Management System

Biomedical researchers often work with massive, detailed and heterogeneous datasets. These datasets raise new challenges of information organization and management for scientific interpretation, as they demand much of the researchers' time and attention. The current study investigated the nature of the problems that researchers face when dealing with such data. Four major problems identified with existing biomedical scientific information management methods were related to data organization, data sharing, collaboration, and publications. Therefore, there is a compelling need to develop an efficient and user-friendly information management system to handle the biomedical research data. This study evaluated the implementation of an information management system, which was introduced as part of the collaborative research to increase scientific productivity in a research laboratory. Laboratory members seemed to exhibit frustration during the implementation process. However, empirical findings revealed that they gained new knowledge and completed specified tasks while working together with the new system. Hence, researchers are urged to persist and persevere when dealing with any new technology, including an information management system in a research laboratory environment.

Different usage of the same oncology information system in two hospitals in Sydney--lessons go beyond the initial introduction

BACKGROUND AND PURPOSE

The experience of clinicians at two public hospitals in Sydney, Australia, with the introduction and use of an oncology information system (OIS) was examined to extract lessons to guide the introduction of clinical information systems in public hospitals.

METHODS AND MATERIALS

Semi-structured interviews were conducted with 12 of 15 radiation oncologists employed at the two hospitals. The personnel involved in the decision making process for the introduction of the system were contacted and their decision making process revisited. The transcribed data were analyzed using NVIVO software. Themes emerged included implementation strategies and practices, the radiation oncologists' current use and satisfaction with the OIS, project management and the impact of the OIS on clinical practice.

RESULTS

The hospitals had contrasting experiences in their introduction and use of the OIS. Hospital A used the OIS in all aspects of clinical documentation. Its implementation was associated with strong advocacy by the Head of Department, input by a designated project manager, and use and development of the system by all staff, with timely training and support. With no vision of developing a paperless information system, Hospital B used the OIS only for booking and patient tracking. A departmental policy that data entry for the OIS was centrally undertaken by administrative staff distanced clinicians from the system. All the clinicians considered that the OIS should continuously evolve to meet changing clinical needs and departmental quality improvement initiatives.

CONCLUSIONS

This case study indicates that critical factors for the successful introduction of clinical information systems into hospital environment were an initial clear vision to be paperless, strong clinical leadership and management at the departmental level, committed project management, and involvement of all staff, with appropriate training. Clinician engagement is essential for post-adoption evolution of clinical information systems.

Translational informatics: enabling high-throughput research paradigms

A common thread throughout the clinical and translational research domains is the need to collect, manage, integrate, analyze, and disseminate large-scale, heterogeneous biomedical data sets. However, well-established and broadly adopted theoretical and practical frameworks and models intended to address such needs are conspicuously absent in the published literature or other reputable knowledge sources. Instead, the development and execution of multidisciplinary, clinical, or translational studies are significantly limited by the propagation of "silos" of both data and expertise. Motivated by this fundamental challenge, we report upon the current state and evolution of biomedical informatics as it pertains to the conduct of high-throughput clinical and translational research and will present both a conceptual and practical framework for the design and execution of informatics-enabled studies. The objective of presenting such findings and constructs is to provide the clinical and translational research community with a common frame of reference for discussing and expanding upon such models and methodologies.

Health IT success and failure: recommendations from literature and an AMIA workshop

With the United States joining other countries in national efforts to reap the many benefits that use of health information technology can bring for health care quality and savings, sobering reports recall the complexity and difficulties of implementing even smaller-scale systems. Despite best practice research that identified success factors for health information technology projects, a majority, in some sense, still fail. Similar problems plague a variety of different kinds of applications, and have done so for many years. Ten AMIA working groups sponsored a workshop at the AMIA Fall 2006 Symposium. It was entitled "Avoiding The F-Word: IT Project Morbidity, Mortality, and Immortality" and focused on this under-addressed problem. PARTICIPANTS discussed communication, workflow, and quality; the complexity of information technology undertakings; the need to integrate all aspects of projects, work environments, and regulatory and policy requirements; and the difficulty of getting all the parts and participants in harmony. While recognizing that there still are technical issues related to functionality and interoperability, discussion affirmed the emerging consensus that problems are due to sociological, cultural, and financial issues, and hence are more managerial than technical. Participants drew on lessons from experience and research in identifying important issues, action items, and recommendations to address the following: what "success" and "failure" mean, what contributes to making successful or unsuccessful systems, how to use failure as an enhanced learning opportunity for continued improvement, how system successes or failures should be studied, and what AMIA should do to enhance opportunities for successes. The workshop laid out a research agenda and recommended action items, reflecting the conviction that AMIA members and AMIA as an organization can take a leadership role to make projects more practical and likely to succeed in health care settings.

Clinical research informatics: challenges, opportunities and definition for an emerging domain

OBJECTIVES

Clinical Research Informatics, an emerging sub-domain of Biomedical Informatics, is currently not well defined. A formal description of CRI including major challenges and opportunities is needed to direct progress in the field.

DESIGN

Given the early stage of CRI knowledge and activity, we engaged in a series of qualitative studies with key stakeholders and opinion leaders to determine the range of challenges and opportunities facing CRI. These phases employed complimentary methods to triangulate upon our findings.

MEASUREMENTS

Study phases included: 1) a group interview with key stakeholders, 2) an email follow-up survey with a larger group of self-identified CRI professionals, and 3) validation of our results via electronic peer-debriefing and member-checking with a group of CRI-related opinion leaders. Data were collected, transcribed, and organized for formal, independent content analyses by experienced qualitative investigators, followed by an iterative process to identify emergent categorizations and thematic descriptions of the data.

RESULTS

We identified a range of challenges and opportunities facing the CRI domain. These included 13 distinct themes spanning academic, practical, and organizational aspects of CRI. These findings also informed the development of a formal definition of CRI and supported further representations that illustrate areas of emphasis critical to advancing the domain.

CONCLUSIONS

CRI has emerged as a distinct discipline that faces multiple challenges and opportunities. The findings presented summarize those challenges and opportunities and provide a framework that should help inform next steps to advance this important new discipline.

Commentaries on "Informatics and medicine: from molecules to populations"

OBJECTIVE

To discuss interdisciplinary research and education in the context of informatics and medicine by commenting on the paper of Kuhn et al. "Informatics and Medicine: From Molecules to Populations".

METHOD

Inviting an international group of experts in biomedical and health informatics and related disciplines to comment on this paper.

RESULTS AND CONCLUSIONS

The commentaries include a wide range of reasoned arguments and original position statements which, while strongly endorsing the educational needs identified by Kuhn et al., also point out fundamental challenges that are very specific to the unusual combination of scientific, technological, personal and social problems characterizing biomedical informatics. They point to the ultimate objectives of managing difficult human health problems, which are unlikely to yield to technological solutions alone. The psychological, societal, and environmental components of health and disease are emphasized by several of the commentators, setting the stage for further debate and constructive suggestions.