Data management in clinical research: An overview

Defining and Developing a Generic Framework for Monitoring Data Quality in Clinical Research

Evidence for the need for high data quality in clinical research is well established. The rigor of clinical research conclusions rely heavily on good quality data, which relies on good documentation practices. Little attention has been given to clear guidelines and definitions to monitor data quality. To address this, a "fit-for-use" data quality monitoring framework (DQMF) for clinical research was developed based on a holistic design-oriented approach. An integrated literature review and feasibility study underpinned the framework development. Ontology of key terms, concepts, methods, and standards were recorded using a consensus approach and mind mapping technique. The DQMF is presented as a nested concentric network illustrating concept relationships and hierarchy. Face validation was conducted, and common terminology and definitions are listed. The consolidated DQMF can be adapted according to study context and data availability aiding in the development of a long-term strategy with increased efficacy for clinical data quality monitoring.

How to set up a database?-a five-step process

Database set-up directly impacts the quality and viability of research data, and therefore is a crucial part of the quality of clinical research. Setting up a quality database implies following a strict data-management process. Too much collected information threatens the quality of the information required to achieve the objectives of the study. Therefore, the data that will be collected and managed have to be cautiously discussed and selected. Case report forms (CRF) are the tools the most frequently used to collect the data specified by the protocol. An informative and well-structured document simplifies database design and data validation. Key elements are about choice of sequential or thematic structuring, information and type of information that should be entered and the importance of data standards and coding guide. Final database must be structured with unique ID patient, with one record per subject or per measure. Specific information must be provided for each variable according to the database specifications. The quality of the results is directly related to the quality of the collected data. The CRF should then be completed as fully and accurately as possible. Data validation relies on three key points: the CRF completion guidelines, the Edit Checks process and the Data clarification process. Various open source or business software applications provide all functionalities to set up a clinical data base and CRF. The General Data Protection Regulation (GDPR) standardizes and strengthens the protection of personal data across the EU and for other country's data being "processed" within the EU. The General principles include lawfulness, fairness and transparency, restricted use of data, data minimization, accuracy, limited storage, confidentiality and probity, and accountability.

A New Software Platform to Improve Multidisciplinary Tumor Board Workflows and User Satisfaction: A Pilot Study

Background

Workflow and preparation for holding multidisciplinary cancer case reviews (i.e., Tumor Boards) is time-consuming and cumbersome. Use of a software platform might improve this process. This pilot study assessed the impact of a new software platform on tumor board preparation workflow and user satisfaction compared to current methods.

Materials and Methods

Using current methods and the NAVIFY Tumor Board Solution, this study assessed the number of tasks and time to prepare tumor board cases. Participants completed online surveys assessing ease of use and satisfaction with current and new platforms.

Results

A total of 41 sessions included two surgeons, two oncologists, two pathologists, and two radiologists preparing tumor board cases with 734 tasks were recorded. Overall, there was no difference in the number of tasks using either preparation method (341 current, 393 NAVIFY Tumor Board solution). There was a significant difference in overall preparation time as a function of specialty (F = 71.74, P < 0.0001), with oncologists, radiologists, and surgeons having reduced times with NAVIFY Tumor Board solution compared to the current platform and pathologists having equivalent times. There was a significant difference (F = 38.98, P < 0.0001) for times as a function of task category. Review of clinical course data and other preparation tasks decreased significantly, but pathology and radiology review did not differ significantly. The new platform received higher ratings than the current methods on all survey questions regarding the ease of use and satisfaction.

Conclusions

The study supported the hypothesis that the new software platform can improve Tumor Board preparation. Further study is needed to assess the impact of this platform in different hospitals, different data storage systems, with different observers, and different types of Tumor board cases as well as its impact on the quality of the tumor board discussion.

Combining Adobe Forms and Dropbox to Obtain a Low-Cost Electronic Data Collection System

BACKGROUND

The method used to collect, store, and analyze data can affect the quality of data obtained. Many studies have shown the advantages of electronic systems over paper. Numerous tools are available for this purpose; however, they tend to be implemented only in high-budget protocols as they tend to be expensive.

INTRODUCTION

We report the experience of the Affective Disorders' Clinic of the National Institute of Psychiatry in Mexico City using Adobe Acrobat^® Forms combined with Dropbox^® as storage and distribution method for clinical protocols.

MATERIALS AND METHODS

Adobe Acrobat 9 Pro^® was used to create personalized forms to capture data for seven different clinical purposes. Dropbox was used to share the distributed forms with fellow researchers. Completed forms were stored in Dropbox and the main researcher kept the data collection file, from where the forms were gathered for analysis.

RESULTS

Sixteen forms created for different clinical purposes had an average of 1.2% missing fields.

DISCUSSION

The main drawbacks relate to remote filling and the time spent in fine-tuning complex forms. The main advantages relate to the simplicity of the implementation of the method and the fact that no software other than the one available at the Institute is required.

CONCLUSIONS

Compared with other types of electronic data collection systems, the combination of these tools achieves similar goals and is low cost, requiring minimal knowledge of informatics.

Baseline Assessment of the Evolving 2017 eClinical Landscape

The volume and diversity of data collected to support each clinical study has increased dramatically in response to the rising scope and complexity of global drug development programs. The Tufts Center for the Study of Drug Development conducted an online survey of 257 unique global companies-77% drug development sponsors and 23% contract service providers-to assess clinical data management practices and experiences. Study results indicate that companies are using an average of 6 different applications to support each clinical study and that companies are collecting a range of data types including that from case report forms, lab procedures, pharmacokinetics, biomarker, outcomes assessment, mobile health, and social media. Companies report that the primary electronic data capture (EDC) is capturing traditional data types but not many of the newer ones. Respondents report spending an average of 68.3 days to build and release a study database, 8.1 days between the patient visit and when that patient's data are entered into the EDC system, and 36.3 days on average to lock the database following the last patient last visit. Average cycle time durations are longer and more variable than those observed ten years ago. Subgroup differences (eg, by company size and company type) and factors contributing to data management cycle time and experience are discussed.

Exploring Data Quality Management within Clinical Trials

BACKGROUND

Clinical trials are an important research method for improving medical knowledge and patient care. Multiple international and national guidelines stipulate the need for data quality and assurance. Many strategies and interventions are developed to reduce error in trials, including standard operating procedures, personnel training, data monitoring, and design of case report forms. However, guidelines are nonspecific in the nature and extent of necessary methods.

OBJECTIVE

This article gathers information about current data quality tools and procedures used within Australian clinical trial sites, with the aim to develop standard data quality monitoring procedures to ensure data integrity.

METHODS

Relevant information about data quality management methods and procedures, error levels, data monitoring, staff training, and development were collected. Staff members from 142 clinical trials listed on the National Health and Medical Research Council (NHMRC) clinical trials Web site were invited to complete a short self-reported semiquantitative anonymous online survey.

RESULTS

Twenty (14%) clinical trials completed the survey. Results from the survey indicate that procedures to ensure data quality varies among clinical trial sites. Centralized monitoring (65%) was the most common procedure to ensure high-quality data. Ten (50%) trials reported having a data management plan in place and two sites utilized an error acceptance level to minimize discrepancy, set at <5% and 5 to 10%, respectively. The quantity of data variables checked (10-100%), the frequency of visits (once-a-month to annually), and types of variables (100%, critical data or critical and noncritical data audits) for data monitoring varied among respondents. The average time spent on staff training per person was 11.58 hours over a 12-month period and the type of training was diverse.

CONCLUSION

Clinical trial sites are implementing ad hoc methods pragmatically to ensure data quality. Findings highlight the necessity for further research into "standard practice" focusing on developing and implementing publically available data quality monitoring procedures.

Virtual Reality Training for Upper Extremity in Subacute Stroke (VIRTUES)

Objective:

To compare the effectiveness of upper extremity virtual reality rehabilitation training (VR) to time-matched conventional training (CT) in the subacute phase after stroke.

Methods:

In this randomized, controlled, single-blind phase III multicenter trial, 120 participants with upper extremity motor impairment within 12 weeks after stroke were consecutively included at 5 rehabilitation institutions. Participants were randomized to either VR or CT as an adjunct to standard rehabilitation and stratified according to mild to moderate or severe hand paresis, defined as ≥20 degrees wrist and 10 degrees finger extension or less, respectively. The training comprised a minimum of sixteen 60-minute sessions over 4 weeks. The primary outcome measure was the Action Research Arm Test (ARAT); secondary outcome measures were the Box and Blocks Test and Functional Independence Measure. Patients were assessed at baseline, after intervention, and at the 3-month follow-up.

Results:

Mean time from stroke onset for the VR group was 35 (SD 21) days and for the CT group was 34 (SD 19) days. There were no between-group differences for any of the outcome measures. Improvement of upper extremity motor function assessed with ARAT was similar at the postintervention (p = 0.714) and follow-up (p = 0.777) assessments. Patients in VR improved 12 (SD 11) points from baseline to the postintervention assessment and 17 (SD 13) points from baseline to follow-up, while patients in CT improved 13 (SD 10) and 17 (SD 13) points, respectively. Improvement was also similar for our subgroup analysis with mild to moderate and severe upper extremity paresis.

Conclusions:

Additional upper extremity VR training was not superior but equally as effective as additional CT in the subacute phase after stroke. VR may constitute a motivating training alternative as a supplement to standard rehabilitation.

ClinicalTrials.gov identifier:

NCT02079103.

Classification of evidence:

This study provides Class I evidence that for patients with upper extremity motor impairment after stroke, compared to conventional training, VR training did not lead to significant differences in upper extremity function improvement.

A Web-Based Data Collection Platform for Multisite Randomized Behavioral Intervention Trials: Development, Key Software Features, and Results of a User Survey

Background

Meticulous tracking of study data must begin early in the study recruitment phase and must account for regulatory compliance, minimize missing data, and provide high information integrity and/or reduction of errors. In behavioral intervention trials, participants typically complete several study procedures at different time points. Among HIV-infected patients, behavioral interventions can favorably affect health outcomes. In order to empower newly diagnosed HIV positive individuals to learn skills to enhance retention in HIV care, we developed the behavioral health intervention Integrating ENGagement and Adherence Goals upon Entry (iENGAGE) funded by the National Institute of Allergy and Infectious Diseases (NIAID), where we deployed an in-clinic behavioral health intervention in 4 urban HIV outpatient clinics in the United States. To scale our intervention strategy homogenously across sites, we developed software that would function as a behavioral sciences research platform.

Objective

This manuscript aimed to: (1) describe the design and implementation of a Web-based software application to facilitate deployment of a multisite behavioral science intervention; and (2) report on results of a survey to capture end-user perspectives of the impact of this platform on the conduct of a behavioral intervention trial.

Methods

In order to support the implementation of the NIAID-funded trial iENGAGE, we developed software to deploy a 4-site behavioral intervention for new clinic patients with HIV/AIDS. We integrated the study coordinator into the informatics team to participate in the software development process. Here, we report the key software features and the results of the 25-item survey to evaluate user perspectives on research and intervention activities specific to the iENGAGE trial (N=13).

Results

The key features addressed are study enrollment, participant randomization, real-time data collection, facilitation of longitudinal workflow, reporting, and reusability. We found 100% user agreement (13/13) that participation in the database design and/or testing phase made it easier to understand user roles and responsibilities and recommended participation of research teams in developing databases for future studies. Users acknowledged ease of use, color flags, longitudinal work flow, and data storage in one location as the most useful features of the software platform and issues related to saving participant forms, security restrictions, and worklist layout as least useful features.

Conclusions

The successful development of the iENGAGE behavioral science research platform validated an approach of early and continuous involvement of the study team in design development. In addition, we recommend post-hoc collection of data from users as this led to important insights on how to enhance future software and inform standard clinical practices.

Trial Registration

Clinicaltrials.gov NCT01900236; (https://clinicaltrials.gov/ct2/show/NCT01900236 (Archived by WebCite at http://www.webcitation.org/6qAa8ld7v)

A SOA-Based Platform to Support Clinical Data Sharing

The eSource Data Interchange Group, part of the Clinical Data Interchange Standards Consortium, proposed five scenarios to guide stakeholders in the development of solutions for the capture of eSource data. The fifth scenario was subdivided into four tiers to adapt the functionality of electronic health records to support clinical research. In order to develop a system belonging to the "Interoperable" Tier, the authors decided to adopt the service-oriented architecture paradigm to support technical interoperability, Health Level Seven Version 3 messages combined with LOINC (Logical Observation Identifiers Names and Codes) vocabulary to ensure semantic interoperability, and Healthcare Services Specification Project standards to provide process interoperability. The developed architecture enhances the integration between patient-care practice and medical research, allowing clinical data sharing between two hospital information systems and four clinical data management systems/clinical registries. The core is formed by a set of standardized cloud services connected through standardized interfaces, involving client applications. The system was approved by a medical staff, since it reduces the workload for the management of clinical trials. Although this architecture can realize the "Interoperable" Tier, the current solution actually covers the "Connected" Tier, due to local hospital policy restrictions.

Development of an automated appendix generation system (ARGUS) for clinical study reports

Data handling and tabulation are a time-consuming job when writing appendices for clinical study reports. The authors have developed an automated appendix generation system (ARGUS) conforming to the CDISC/SDTM standard using SAS (version 9.3) and R (version 3.3.1: for PK plot generation). It consists of the one main program and three subprograms. The program runs to convert a database file into an appendix document with about 100 tables and plots in MS Word format within one min after pressing the submit button under common desktop environments. We found that tasks of constructing appendices for a typical 2×2 crossover design study that have taken our team about 8 days were completed within 6 or 7 hours using the ARGUS system.

Federating Heterogeneous Datasets to Enhance Data Sharing and Experiment Reproducibility

Recent studies have demonstrated the difficulties to replicate scientific findings and/or experiments published in past.¹ The effects seen in the replicated experiments were smaller than previously reported. Some of the explanations for these findings include the complexity of the experimental design and the pressure on researches to report positive findings. The International Committee of Medical Journal Editors (ICMJE) suggests that every study considered for publication must submit a plan to share the de-identified patient data no later than 6 months after publication. There is a growing demand to enhance the management of clinical data, facilitate data sharing across institutions and also to keep track of the data from previous experiments. The ultimate goal is to assure the reproducibility of experiments in the future. This paper describes Shiny-tooth, a web based application created to improve clinical data acquisition during the clinical trial; data federation of such data as well as morphological data derived from medical images; Currently, this application is being used to store clinical data from an osteoarthritis (OA) study. This work is submitted to the SPIE Biomedical Applications in Molecular, Structural, and Functional Imaging conference.

ODaCCI: Ontology-guided Data Curation for Multisite Clinical Research Data Integration in the NINDS Center for SUDEP Research

Sudden Unexpected Death in Epilepsy (SUDEP) is the leading mode of epilepsy-related death. The Center for SUDEP Research (CSR) is an NINDS-funded Center Without Wall's initiative aimed at prospectively creating a comprehensive clinical research resource for SUDEP. This resource consists of a growing set of data and biological samples of a statistically significant cohort of patients at an elevated risk, best represented by the Epilepsy Monitoring Unit (EMU) patient population. The Informatics and Data Analytics Core (IDAC) of CSR has developed a state-of-the- art informatics infrastructure, to integrate patient data captured in multiple EMU's at a greatly accelerated pace. Data quality assurance is a priority of IDAC. This paper reports our approach, Ontology-guided Data Curation for Multisite Clinical Research Data Integration (ODaCCI), to address the challenging task of centralized data curation while new data is continuously generated and integrated from distributed sites. ODaCCI leverages the Epilepsy and Seizure Ontology not only for upstream data capture, but also for supporting a range of quality assurance tasks such as data quality monitoring, data update, and data reports. Between October 2014 and February 2016, ODaCCI has integrated phenotypic and electroencephalogram signal data of 629 patients from 7 clinical sites, while supporting continuous and asynchronous data quality enhancement overtime.

Radiological clinical trials: Proposal of a problem-finding questionnaire to improve study success

AIM

To develop a survey to help define the main problems in radiological clinical trials.

METHODS

Since 2006, we have managed seven different radiological clinical trials recruiting patients in academic and non-academic centres. We developed a preliminary questionnaire using a four-round Delphi approach to identify problems occurring in radiological clinical trials run at our centre. We investigated the recruitment experience, involvement of all multi-disciplinary team members and main obstacles to completing the projects. A final round of Delphi processes elucidated solutions to the identified problems.

RESULTS

Among 19/20 (95%) respondents, 10 (53%) were young physicians (under 35 years old), and the respondents included non-faculty members, fellows, residents, and undergraduate students. Ninety-four percent (18/19) of respondents showed interest in conducting clinical trials. On a scale of 1 to 10, the problems with higher/worse scores (8-9) were related to technical or communication problems. The most frequent problems across all studies were technical problems related to clinical trial equipment, insufficient willingness to participate, obstacles to understanding the design of electronic-case report form and extra work.

CONCLUSION

The developed questionnaire identified the main recurring problems in radiological clinical trials as perceived by end-users and helped define possible solutions that are mostly related to having dedicated clinical trial research staff.

Concept and implementation of a study dashboard module for a continuous monitoring of trial recruitment and documentation

BACKGROUND

The difficulty of managing patient recruitment and documentation for clinical trials prompts a demand for instruments for closely monitoring these critical but unpredictable processes. Increasingly adopted Electronic Data Capture (EDC) applications provide novel opportunities to reutilize stored information for an efficient management of traceable trial workflows. In related clinical and administrative settings, so-called digital dashboards that continuously visualize time-dependent parameters have recently met a growing acceptance. To investigate the technical feasibility of a study dashboard for monitoring the progress of patient recruitment and trial documentation, we set out to develop a propositional prototype in the form of a separate software module.

METHODS

After narrowing down functional requirements in semi-structured interviews with study coordinators, we analyzed available interfaces of a locally deployed EDC application, and designed the prototypical study dashboard based on previous findings. The module thereby leveraged a standardized export format in order to extract and import relevant trial data into a clinical data warehouse. Web-based reporting tools then facilitated the definition of diverse views, including diagrams of the progress of patient accrual and form completion at different granularity levels. To estimate the utility of the dashboard and its compatibility with current workflows, we interviewed study coordinators after a demonstration of sample outputs from ongoing trials.

RESULTS

The employed tools promoted a rapid development. Displays of the implemented dashboard are organized around an entry page that integrates key metrics for available studies, and which links to more detailed information such as study-specific enrollment per center. The interviewed experts commented that the included graphical summaries appeared suitable for detecting that something was generally amiss, although practical remedies would mostly depend on additional information such as access to the original patient-specific data. The dependency on a separate application was seen as a downside. Interestingly, the prospective users warned that in some situations knowledge of specific accrual statistics might undermine blinding in a subtle yet intricate fashion, so ignorance of certain patient features was seen as sometimes preferable for reproducibility.

DISCUSSION

Our proposed study dashboard graphically recaps key progress indicators of patient accrual and trial documentation. The modular implementation illustrates the technical feasibility of the approach. The use of a study dashboard might introduce certain technical requirements as well as subtle interpretative complexities, which may have to be weighed against potential efficiency gains.

Improving Quality and Safety Through Use of Secondary Data: Methods Case Study

Researchers need to evaluate the strengths and weaknesses of data sets to choose a secondary data set to use for a health care study. This research method review informs the reader of the major issues necessary for investigators to consider while incorporating secondary data into their repertoire of potential research designs and shows the range of approaches the investigators may take to answer nursing research questions in a variety of context areas. The researcher requires expertise in locating and judging data sets and in the development of complex data management skills for managing large numbers of records. There are important considerations such as firm knowledge of the research question supported by the conceptual framework and the selection of appropriate databases, which guide the researcher in delineating the unit of analysis. Other more complex issues for researchers to consider when conducting secondary data research methods include data access, management and security, and complex variable construction.

Centralized Data Management in a Multicountry, Multisite Population-based Study

BACKGROUND

A centralized data management system was developed for data collection and processing for the Aetiology of Neonatal Infection in South Asia (ANISA) study. ANISA is a longitudinal cohort study involving neonatal infection surveillance and etiology detection in multiple sites in South Asia. The primary goal of designing such a system was to collect and store data from different sites in a standardized way to pool the data for analysis.

METHODS

We designed the data management system centrally and implemented it to enable data entry at individual sites. This system uses validation rules and audit that reduce errors. The study sites employ a dual data entry method to minimize keystroke errors. They upload collected data weekly to a central server via internet to create a pooled central database. Any inconsistent data identified in the central database are flagged and corrected after discussion with the relevant site. The ANISA Data Coordination Centre in Dhaka provides technical support for operations, maintenance and updating the data management system centrally. Password-protected login identifications and audit trails are maintained for the management system to ensure the integrity and safety of stored data.

CONCLUSION

Centralized management of the ANISA database helps to use common data capture forms (DCFs), adapted to site-specific contextual requirements. DCFs and data entry interfaces allow on-site data entry. This reduces the workload as DCFs do not need to be shipped to a single location for entry. It also improves data quality as all collected data from ANISA goes through the same quality check and cleaning process.

Use of the data system for field management of a clinical study conducted in Kolkata, India

Background

Designing an appropriate data system is important to the success of a clinical study. However, little information is available on this topic. We share our experiences on designing, developing, and implementation of a data system for management of data and field activities of a complex clinical study.

Methods

The data system was implemented aiming at determining the biological basis for the underperformance of oral vaccines, such as polio and rotavirus vaccines in children at a site in Kolkata, India. The system included several functionalities to control data and field activities. It was restricted to authorized users based on their access privileges. A relational database platform was chosen, and Microsoft Visual FoxPro 7.0 (Microsoft Corporation, Seattle, WA, USA) was used to develop the system. The system was installed at the clinic and data office to facilitate both the field and data management activities.

Results

Data were doubly entered by two different data operators to identify keypunching errors in the data. Outliers, duplication, inconsistencies, missing entries, and linkage were also checked. Every modification and users log-in/log-out information was auto-recorded in an audit trail. The system offered tools for preparation of visit schedule of the participants. A visit considered as protocol deviation was documented by the system. The system alerted field staff to every upcoming visit date to organize the field activities and to inform participants which day to come. The system also produced a growth chart for evaluating nutritional status and referring the child to a specialized clinic if found to be severely malnourished.

Conclusion

The data system offered unique features for controlling for both data and field activities, which led to minimize drop-out rates as well as protocol deviations. Such system is warranted for a successful clinical study.

Paperless clinical trials: Myth or reality?

There is an urgent need to expedite the time-to-market for new drugs and to make the approval process simpler. But clinical trials are a complex process and the increased complexity leads to decreased efficiency. Hence, pharmaceutical organizations want to move toward a more technology-driven clinical trial process for recording, analyzing, reporting, archiving, etc., In recent times, the progress has certainly been made in developing paperless systems that improve data capture and management. The adaptation of paperless processes may require major changes to existing procedures. But this is in the best interests of these organizations to remain competitive because a paperless clinical trial would lead to a consistent and streamlined framework. Moreover, all major regulatory authorities also advocate adoption of paperless trial. But challenges still remain toward implementation of paperless clinical trial process.

Online electronic data capture and research data repository system for clinical and translational research

Data is at the core of any clinical and translational research (CTR). In many studies, the electronic data capture (EDC) method has been found to be more efficient than standard paper-based data collection methods in many aspects, including accuracy, integrity, timeliness, and cost-effectiveness. The objective of this article is to present a secure, web-based EDC system for CTR that has been implemented by the Institute for Clinical and Translational Science (iCATS) at the University of Missouri School of Medicine.

The practice of 'doing' evaluation: lessons learned from nine complex intervention trials in action

Background

There is increasing recognition among trialists of the challenges in understanding how particular ‘real-life’ contexts influence the delivery and receipt of complex health interventions. Evaluations of interventions to change health worker and/or patient behaviours in health service settings exemplify these challenges. When interpreting evaluation data, deviation from intended intervention implementation is accounted for through process evaluations of fidelity, reach, and intensity. However, no such systematic approach has been proposed to account for the way evaluation activities may deviate in practice from assumptions made when data are interpreted.

Methods

A collective case study was conducted to explore experiences of undertaking evaluation activities in the real-life contexts of nine complex intervention trials seeking to improve appropriate diagnosis and treatment of malaria in varied health service settings. Multiple sources of data were used, including in-depth interviews with investigators, participant-observation of studies, and rounds of discussion and reflection.

Results and discussion

From our experiences of the realities of conducting these evaluations, we identified six key ‘lessons learned’ about ways to become aware of and manage aspects of the fabric of trials involving the interface of researchers, fieldworkers, participants and data collection tools that may affect the intended production of data and interpretation of findings. These lessons included: foster a shared understanding across the study team of how individual practices contribute to the study goals; promote and facilitate within-team communications for ongoing reflection on the progress of the evaluation; establish processes for ongoing collaboration and dialogue between sub-study teams; the importance of a field research coordinator bridging everyday project management with scientific oversight; collect and review reflective field notes on the progress of the evaluation to aid interpretation of outcomes; and these approaches should help the identification of and reflection on possible overlaps between the evaluation and intervention.

Conclusion

The lessons we have drawn point to the principle of reflexivity that, we argue, needs to become part of standard practice in the conduct of evaluations of complex interventions to promote more meaningful interpretations of the effects of an intervention and to better inform future implementation and decision-making.

Design and evaluation of a multimedia electronic patient record "oncoflow" with clinical workflow assistance for head and neck tumor therapy

OBJECTIVES

The management of patient-specific information is a challenging task for surgeons and physicians because existing clinical information systems are insufficiently integrated into daily clinical routine and contained information entities are distributed across different proprietary databases. Thus, existing information is hardly usable for further electronic processing, workflow support or clinical studies.

METHODS

A Web-based clinical information system has been developed that automatically imports patient-specific information from different information systems. The system is tailored to the existing workflow for the treatment of patients with head and neck cancer. In this paper, the clinical assistance functions and a quantitative as well as a qualitative system evaluation are presented.

RESULTS

The information system has been deployed at a clinical site and is in use in daily clinical routine. Two evaluation studies show that the information integration, the structured information presentation in the Web browser and the assistance functions improve the physician's workflow. The studies also show that the usage of the new information system does not impair the time physicians need for a process step compared with the usage of the existing information system.

CONCLUSIONS

Information integration is crucial for efficient workflow support in the clinic. The central access to information within a modern and structured user interface saves valuable time for the physician. The comprehensive database allows an instant usage of the existing information clinical workflow support or the conduction of trial studies.

Roadmap for the development of the University of North Carolina at Chapel Hill Genitourinary OncoLogy Database--UNC GOLD

BACKGROUND

The management of genitourinary malignancies requires a multidisciplinary care team composed of urologists, medical oncologists, and radiation oncologists. A genitourinary (GU) oncology clinical database is an invaluable resource for patient care and research. Although electronic medical records provide a single web-based record used for clinical care, billing, and scheduling, information is typically stored in a discipline-specific manner and data extraction is often not applicable to a research setting. A GU oncology database may be used for the development of multidisciplinary treatment plans, analysis of disease-specific practice patterns, and identification of patients for research studies. Despite the potential utility, there are many important considerations that must be addressed when developing and implementing a discipline-specific database.

METHODS AND MATERIALS

The creation of the GU oncology database including prostate, bladder, and kidney cancers with the identification of necessary variables was facilitated by meetings of stakeholders in medical oncology, urology, and radiation oncology at the University of North Carolina (UNC) at Chapel Hill with a template data dictionary provided by the Department of Urologic Surgery at Vanderbilt University Medical Center. Utilizing Research Electronic Data Capture (REDCap, version 4.14.5), the UNC Genitourinary OncoLogy Database (UNC GOLD) was designed and implemented.

RESULTS

The process of designing and implementing a discipline-specific clinical database requires many important considerations. The primary consideration is determining the relationship between the database and the Institutional Review Board (IRB) given the potential applications for both clinical and research uses. Several other necessary steps include ensuring information technology security and federal regulation compliance; determination of a core complete dataset; creation of standard operating procedures; standardizing entry of free text fields; use of data exports, queries, and de-identification strategies; inclusion of individual investigators' data; and strategies for prioritizing specific projects and data entry.

CONCLUSIONS

A discipline-specific database requires a buy-in from all stakeholders, meticulous development, and data entry resources to generate a unique platform for housing information that may be used for clinical care and research with IRB approval. The steps and issues identified in the development of UNC GOLD provide a process map for others interested in developing a GU oncology database.

Achieving a Culture of Quality in Pharmaceutical Biostatistics: One Approach to Achieving Quality Deliverables

Biostatistics units within the pharmaceutical industry are faced with the challenge of increasing productivity, reducing costs, and accelerating timelines while at the same time maintaining high quality. This paper describes how the global biostatistics unit in one company is addressing this challenge by nurturing a "culture of quality" to engage frontline staff responsible for daily deliverables. The approach ties together the measurement, improvement, and maintenance of quality, thus inviting staff to understand their role in achieving quality deliverables. Appointment within the operations unit of a quality lead, who is responsible for quality oversight, further increases visibility and contributes to the culture. Measurement is addressed through the use of a quarterly quality report card in which quality-related metrics are consolidated and tracked over time. Trends and outliers are explored. Through established process improvement practices, a Lean Sigma team tackles select high-value processes. Quality is maintained through continuous feedback loops driven by formal quality assurance audits and logs of lessons learned. This paper first considers how to define quality in this context; then addresses the sometimes disjointed areas of measurement, improvement, and maintenance; and ultimately ties them together to drive a culture of quality.