Electronic data collection options for practice-based research networks

The Accuracy of the Learning-Curve Cumulative Sum Method in Assessing Brachial Plexus Block Competency

BACKGROUND

The learning-curve cumulative sum method (LC-CUSUM) and its risk-adjusted form (RA-LC-CUSUM) have been proposed as performance-monitoring methods to assess competency during the learning phase of procedural skills. However, scarce data exist about the method's accuracy. This study aimed to compare the accuracy of LC-CUSUM forms using historical data consisting of sequences of successes and failures in brachial plexus blocks (BPBs) performed by anesthesia residents.

METHODS

Using historical data from 1713 BPB performed by 32 anesthesia residents, individual learning curves were constructed using the LC-CUSUM and RA-LC-CUSUM methods. A multilevel logistic regression model predicted the procedure-specific risk of failure incorporated in the RA-LC-CUSUM calculations. Competency was defined as a maximum 15% cumulative failure rate and was used as the reference for determining the accuracy of both methods.

RESULTS

According to the LC-CUSUM method, 22 residents (84.61%) attained competency after a median of 18.5 blocks (interquartile range [IQR], 14-23), while the RA-LC-CUSUM assigned competency to 20 residents (76.92%) after a median of 17.5 blocks (IQR, 14-25, P = .001). The median failure rate at reaching competency was 6.5% (4%-9.75%) under the LC-CUSUM and 6.5% (4%-9%) for the RA-LC-CUSUM method ( P = .37). The sensitivity of the LC-CUSUM (85%; 95% confidence interval [CI], 71%-98%) was similar to the RA-LC-CUSUM method (77%; 95% CI, 61%-93%; P = .15). Identical specificity values were found for both methods (67%; 95% CI, 29%-100%, P = 1).

CONCLUSIONS

The LC-CUSUM and RA-LC-CUSUM methods were associated with substantial false-positive and false-negative rates. Also, small lower limits for the 95% CIs around the accuracy measures were observed, indicating that the methods may be inaccurate for high-stakes decisions about resident competency at BPBs.

Data quality assessment of the Enhanced Gonococcal Antimicrobial Surveillance Programme (EGASP), Thailand, 2015-2021

BACKGROUND

Quality assessments of gonococcal surveillance data are critical to improve data validity and to enhance the value of surveillance findings. Detecting data errors by systematic audits identifies areas for quality improvement. We designed and implemented an internal audit process to evaluate the accuracy and completeness of surveillance data for the Thailand Enhanced Gonococcal Antimicrobial Surveillance Programme (EGASP).

METHODS

We conducted a data quality audit of source records by comparison with the data stored in the EGASP database for five audit cycles from 2015-2021. Ten percent of culture-confirmed cases of Neisseria gonorrhoeae were randomly sampled along with any cases identified with elevated antimicrobial susceptibility testing results and cases with repeat infections. Incorrect and incomplete data were investigated, and corrective action and preventive actions (CAPA) were implemented. Accuracy was defined as the percentage of identical data in both the source records and the database. Completeness was defined as the percentage of non-missing data from either the source document or the database. Statistical analyses were performed using the t-test and the Fisher's exact test.

RESULTS

We sampled and reviewed 70, 162, 85, 68, and 46 EGASP records during the five audit cycles. Overall accuracy and completeness in the five audit cycles ranged from 93.6% to 99.4% and 95.0% to 99.9%, respectively. Overall, completeness was significantly higher than accuracy (p = 0.017). For each laboratory and clinical data element, concordance was >85% in all audit cycles except for two laboratory data elements in two audit cycles. These elements significantly improved following identification and CAPA implementation.

DISCUSSION

We found a high level of data accuracy and completeness in the five audit cycles. The implementation of the audit process identified areas for improvement. Systematic quality assessments of laboratory and clinical data ensure high quality EGASP surveillance data to monitor for antimicrobial resistant Neisseria gonorrhoeae in Thailand.

Evaluating the Efficiency of Survey Collection Methods to Trauma Patients

BACKGROUND

Traumatic injury survivors often experience negative health consequences, impacting recovery. No studies have assessed the feasibility of evaluating the resiliency of hospitalized trauma patients using the 10-item Connor-Davidson Resilience Scale (CD-RISC-10).

PURPOSE

The purpose of this study was to determine the most efficient method to collect survey responses on the CD-RISC-10.

METHODS

This cross-sectional study used a convenience sample of admitted patients with traumatic injury. Patients were randomized to complete the CD-RISC-10 using pen and paper, tablet, or workstation on wheels.

RESULTS

Of the 161 patient surveys, the tablet-based survey took the shortest time to complete (2 minutes, 21 seconds), and the paper survey resulted in the lowest percentage of missed questions (0.5%). Trauma patients reported high levels of resiliency.

CONCLUSION

The CD-RISC-10 can be easily administered to trauma patients. Clinicians should balance efficiency and patient preferences when deciding on a survey collection method.

The Role of Primary Care in Improving Population Health

Policy Points Systems based on primary care have better population health, health equity, and health care quality, and lower health care expenditure. Primary care can be a boundary-spanning force to integrate and personalize the many factors from which population health emerges. Equitably advancing population health requires understanding and supporting the complexly interacting mechanisms by which primary care influences health, equity, and health costs.

Electronic versus paper-based data collection for conducting health-care research: A cost-comparison analysis

Background

Containing expenditure and efficient resource use is essential to limit the increasing costs of health research. Electronic data collection (EDC) is thought to reduce the costs compared to paper-based data collection (PDC).

Objectives

As economic evidence in this area is scanty, especially in low- and middle-income countries, the objectives of the study are to perform an economic evaluation and compare the cost between EDC and PDC.

Methods

A cost-comparison study was conducted to compare between EDC and PDC from the institutional perspective for the year 2018, based on a community-based survey. Step-down cost accounting was adopted with a bottom-up approach for cost estimation. Total and unit costs were estimated with the base case comparison between EDC and PDC while using SPSS software (e-SPSS and p-SPSS, respectively). We conducted scenario analyses based on the usage of different software, R and STATA for both EDC and PDC (e-R, p-R, e-STATA, and p-STATA, respectively). One-way and probabilistic sensitivity analysis (PSA) was performed to examine the robustness of the observed results.

Results

In the base-case analysis, total costs of EDC and PDC were ₹72,617 ($1060.9) and 87,717 ($1281.5), respectively, with estimated cost reduction of ₹15,100 ($220.6). In other scenarios, the estimated cost reduction for e-R, e-STATA, p-R, p-STATA was ₹-274 ($4.0), 98 ($1.4), 14826 ($216.6), and 15,002 ($219.2), respectively, when compared to EDC-SPSS. On one-way and PSA, the results of the cost-comparison analysis were robust.

Conclusion

EDC minimizes institutional cost for conducting health research. This finding will help researchers in efficiently planning for the budget for their research.

Data Collection for the Fourth Multicentre Confidential Enquiry into Perioperative Equine Fatalities (CEPEF4) Study: New Technology and Preliminary Results

Simple Summary

New technologies allow researchers to improve the methods for immediate, accurate data collection, cleaning and analysis, with minimal geographical limitations. Although much has improved in the field of equine anaesthesia in recent years, we are still far from reducing anaesthetic-related mortality in this species in comparison with small animal anaesthesia. The aim of this multicentre study was to probe the usefulness of an internet-based method that utilised an electronic questionnaire and statistical software to show the data and report outcomes from horses undergoing general anaesthesia and certain procedures using standing sedation. Within six months, 8656 cases from 69 centres were collected: 6701 procedures under general anaesthesia and 1955 under standing sedation. The results demonstrated (i) the utility of the method and (ii) that some horses died unexpectedly when undergoing not only general anaesthesia, but also standing sedation. Finally, (iii) we present some descriptive data that outline the current anaesthesia practice compared with the previous CEPEF2. We concluded that our internet-based method is suitable for this type of study. New techniques may reduce the mortality rate. However, the results presented here should be interpreted cautiously as these are only preliminary data with lower numbers than CEPEF2.

Abstract

It is almost 20 years since the largest observational, multicentre study evaluating the risks of mortality associated with general anaesthesia in horses. We proposed an internet-based method to collect data (cleaned and analysed with R) in a multicentre, cohort, observational, analytical, longitudinal and prospective study to evaluate peri-operative equine mortality. The objective was to report the usefulness of the method, illustrated with the preliminary data, including outcomes for horses seven days after undergoing general anaesthesia and certain procedures using standing sedation. Within six months, data from 6701 procedures under general anaesthesia and 1955 standing sedations from 69 centres were collected. The results showed (i) the utility of the method; also, that (ii) the overall mortality rate for general anaesthesia within the seven-day outcome period was 1.0%. In horses undergoing procedures other than exploratory laparotomy for colic (“noncolics”), the rate was lower, 0.6%, and in “colics” it was higher, at 3.4%. For standing sedations, the overall mortality rate was 0.2%. Finally, (iii) we present some descriptive data that demonstrate new developments since the previous CEPEF2. In conclusion, horses clearly still die unexpectedly when undergoing procedures under general anaesthesia or standing sedation. Our method is suitable for case collection for future studies.

Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial

BACKGROUND

Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity.

METHODS

This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics.

RESULTS

All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings  across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F_(1,112) = .15, p = .699), CRF length (F_(2,112) = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors).

CONCLUSIONS

This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality.

TRIAL REGISTRATION

Clinical Trials NCT02649907 .

Evaluating the feasibility of using online software to collect patient information in a chiropractic practice-based research network

BACKGROUND

Practice based research networks (PBRNs) are increasingly used as a tool for evidence based practice. We developed and tested the feasibility of using software to enable online collection of patient data within a chiropractic PBRN to support clinical decision making and research in participating clinics.

PURPOSE

To assess the feasibility of using online software to collect quality patient information.

METHODS

The study consisted of two phases: 1) Assessment of the quality of information provided, using a standardized form; and 2) Exploration of patients' perspectives and experiences regarding online information provision through semi-structured interviews. Data analysis was descriptive.

RESULTS

Forty-five new patients were recruited. Thirty-six completed online forms, which were submitted by an appropriate person 100% of the time, with an error rate of less than 1%, and submitted in a timely manner 83% of the time. Twenty-one participants were interviewed. Overall, online forms were preferred given perceived security, ease of use, and enabling provision of more accurate information.

CONCLUSIONS

Use of online software is feasible, provides high quality information, and is preferred by most participants. A pen-and-paper format should be available for patients with this preference and in case of technical difficulties.

Implementation of a health data-sharing infrastructure across diverse primary care organizations

Practice-based research networks bring together academic researchers and primary care clinicians to conduct research that improves health outcomes in real-world settings. The Washington, Wyoming, Alaska, Montana, and Idaho region Practice and Research Network implemented a health data-sharing infrastructure across 9 clinics in 3 primary care organizations. Following implementation, we identified challenges and solutions. Challenges included working with diverse primary care organizations, adoption of health information data-sharing technology in a rapidly changing local and national landscape, and limited resources for implementation. Overarching solutions included working with a multidisciplinary academic implementation team, maintaining flexibility, and starting with an established network for primary care organizations. Approaches outlined may generalize to similar initiatives and facilitate adoption of health data sharing in other practice-based research networks.

Customized-Language Voice Survey on Mobile Devices for Text and Image Data Collection Among Ethnic Groups in Thailand: A Proof-of-Concept Study

Background

Public health surveys are often conducted using paper-based questionnaires. However, many problems are associated with this method, especially when collecting data among ethnic groups who speak a different language from the survey interviewer. The process can be time-consuming and there is the risk of missing important data due to incomplete surveys.

Objective

This study was conducted as a proof-of-concept to develop a new electronic tool for data collection, and compare it with standard paper-based questionnaire surveys using the research setting of assessing Knowledge Attitude and Practice (KAP) toward the Expanded Program on Immunization (EPI) among 6 ethnic groups in Chiang Rai Province, Thailand. The two data collection methods were compared on data quality in terms of data completeness and time consumed in collecting the information. In addition, the initiative assessed the participants’ satisfaction toward the use of a smartphone customized-language voice-based questionnaire in terms of perceived ease of use and perceived usefulness.

Methods

Following a cross-over design, all study participants were interviewed using two data collection methods after a one-week washout period. Questions in the paper-based questionnaires in Thai language were translated to each ethnic language by the interviewer/translator when interviewing the study participant. The customized-language voice-based questionnaires were programmed to a smartphone tablet in six, selectable dialect languages and used by the trained interviewer when approaching participants.

Results

The study revealed positive data quality outcomes when using the smartphone, voice-based questionnaire survey compared with the paper-based questionnaire survey, both in terms of data completeness and time consumed in data collection process. Since the smartphone questionnaire survey was programmed to ask questions in sequence, no data was missing and there were no entry errors. Participants had positive attitudes toward answering the smartphone questionnaire; 69% (48/70) reported they understood the questions easily, 71% (50/70) found it convenient, and 66% (46/70) reported a reduced time in data collection. The smartphone data collection method was acceptable by both the interviewers and by the study participants of different ethnicities.

Conclusions

To our knowledge, this is the first study showing that the application of specific features of mobile devices like smartphone tablets (including dropdown choices, capturing pictures, and voiced questions) can be successfully used for data collection. The mobile device can be effectively used for capturing photos of secondary data and collecting primary data with customized-language and voiced questionnaire survey. Using smartphone questionnaires can minimize or eliminate missing data and reduce the time consumed during the data collection process. Smartphone customized-language, voice-based questionnaires for data collection can be an alternative and better approach than standard translated paper-based questionnaires for public health surveys, especially when collecting data among ethnic and hard-to-reach groups residing in multilanguage-speaking settings.

Electronic dental record use and clinical information management patterns among practitioner-investigators in The Dental Practice-Based Research Network

BACKGROUND

The growing availability of electronic data offers practitioners increased opportunities for reusing clinical data for research and quality improvement. However, relatively little is known about what clinical data practitioners keep on their computers regarding patients.

METHODS

The authors conducted a web-based survey of 991 U.S. and Scandinavian practitioner-investigators (P-Is) in The Dental Practice-Based Research Network to determine the extent of their use of computers to manage clinical information; the type of patient information they kept on paper, a computer or both; and their willingness to reuse electronic dental record (EDR) data for research.

RESULTS

A total of 729 (73.6 percent) of 991 P-Is responded.A total of 73.8 percent of U.S. solo practitioners and 78.7 percent of group practitioners used a computer to manage some patient information, and 14.3 percent and 15.9 percent, respectively, managed all patient information on a computer. U.S. practitioners stored appointments, treatment plans, completed treatment and images electronically most frequently, and the periodontal charting, diagnosis, medical history, progress notes and the chief complaint least frequently.More than 90 percent of Scandinavian practitioners stored all information electronically.A total of 50.8 percent of all P-Is were willing to reuse EDR data for research, and 63.1 percent preferred electronic forms for data collection.

CONCLUSION

The results of this study show that the trend toward increased adoption of EDRs in the United States is continuing, potentially making more data in electronic form available for research. Participants appear to be willing to reuse EDR data for research and to collect data electronically.

CLINICAL IMPLICATIONS

The rising rates of EDR adoption may offer increased opportunities for reusing electronic data for quality improvement and research.

Use of an electronic medical record "dotphrase" data template for a prospective head injury study

Introduction:

The adoption of electronic medical records (EMRs) in emergency departments (EDs) has changed the way that healthcare information is collected, charted, and stored. A challenge for researchers is to determine how EMRs may be leveraged to facilitate study data collection efforts. Our objective is to describe the use of a unique data collection system leveraging EMR technology and to compare data entry error rates to traditional paper data collection.

Methods:

This was a retrospective review of data collection methods during a multicenter study of ED, anti-coagulated, head injury patients. On-shift physicians at 4 centers enrolled patients and prospectively completed data forms. These physicians had the option of completing a paper data form or an electronic “dotphrase” (DP) data form. A feature of our Epic®-based EMR is the ability to use DPs to assist in medical information entry. A DP is a preset template that may be inserted into the EMR when the physician types a period followed by a code phrase (in this case “.ichstudy”). Once the study DP was inserted at the bottom of the electronic ED note, it prompted enrolling physicians to answer study questions. Investigators then extracted data directly from the EMR.

Results:

From July 2009 through December 2010, we enrolled 883 patients. DP data forms were used in 288 (32.6%; 95% confidence interval [CI] 29.5, 35.7%) cases and paper data forms in 595 (67.4%; 95% CI 64.3, 70.5%). Sixty-six (43.7%; 95% CI 35.8, 51.6%) of 151 physicians enrolling patients used DP data entry at least once. Using multivariate analysis, we found no association between physician age, gender, or tenure and DP use. Data entry errors were more likely on paper forms (234/595, 39.3%; 95% CI 35.4, 43.3%) than DP forms (19/288, 6.6%; 95% CI 3.7, 9.5%), difference in error rates 32.7% (95% CI 27.9, 37.6%, P < 0.001).

Conclusion:

DP data collection is a feasible means of data collection. DP data forms maintain all study data within the secure EMR environment, obviating the need to maintain and collect paper data forms. This innovation was embraced by many of our emergency physicians and resulted in lower data entry error rates.

An Evaluation of Community Assessment for Public Health Emergency Response (CASPER) in North Carolina, 2003-2010

INTRODUCTION

Community Assessment for Public Health Emergency Response (CASPER) is a group of tools and methods designed by the US Centers for Disease Control and Prevention (CDC) to provide rapid, reliable, and accurate population-based public health information. Since 2003, North Carolina public health professionals have used CASPERs to facilitate public health emergency responses and gather information on other topics including routine community health assessments.

PROBLEM

To date, there has been no evaluation of CASPER use by public health agencies at the state or local level in the US.

METHODS

Local health departments of North Carolina reported when and how CASPERs were used during the period 2003 to 2010 via an online survey. Data on barriers and future plans for using CASPERs also were collected.

RESULTS

Fifty-two of North Carolina's 85 local health departments (61%) completed the survey. Twenty-eight departments reported 46 instances of CASPER use during 2003 to 2010. The majority of CASPERs were performed for community health assessments (n = 20, 43%) or exercises (n = 11, 24%). Fifty-six percent of respondents indicated they were "likely" or "very likely" to use CASPERs in the future; those who had prior experience with CASPERs were significantly more likely (P = .02) to report planned future use of CASPERs compared to those without prior experience with the tool. Lack of training, equipment, and time were the most frequently reported barriers to using CASPERs.

CONCLUSIONS

Local public health agencies with clear objectives and goals can effectively use CASPERs in both routine public health practice and disaster settings.

Practice-based research networks (PBRNs): meeting the challenges of the future

Practice-based research networks (PBRNs) are useful tools for conducting studies in the busy primary care setting, but their continued existence is threatened by a range of challenges. PBRNs must position themselves now to be prepared to face the challenges ahead. For example, experience with the Clinical Translational Science Awards has placed PBRNs at the center of university efforts toward greater community engagement. Networks must use this opportunity to solicit infrastructure support and partner with experienced principal investigators from other disciplines. Successful networks must make greater use of health information technology to solicit clinician involvement, identify and recruit potential subjects, and disseminate key findings. To maintain the active participation of busy clinicians in the clinical research enterprise, networks must find new ways to engage their members and simplify study participation. Networks should pursue clinically relevant projects that create meaning and connect busy practitioners to the larger agenda of primary care research. Finally, collaborating with other networks in a structured and ongoing manner is one way for PBRNs to extend their reach while making maximal use of their unique resources and local expertise.

Practice-based research networks, part I: clinical laboratories to generate and translate research findings into effective patient care

CONTEXT

To improve patient care, athletic training clinicians and researchers should work together to translate research findings into clinical practice. Problems with patient care observed in clinical practice should be translated into research frameworks, where they can be studied. Practice-based research networks (PBRNs) provide a compelling model for linking clinicians and researchers so they can conduct translational research to improve patient care.

OBJECTIVE

To describe (1) the translational research model, (2) practice-based research as a mechanism for translating research findings into clinical practice, (3) the PBRN model and infrastructure, (4) the research potential using the PBRN model, and (5) protection of human participants in PBRN research.

DESCRIPTION

Translational research is the process of transforming research findings into health behavior that ultimately serves the public and attempts to bridge the gap between research and clinical practice. Practice-based research represents the final step in the translational research continuum and describes research conducted by providers in clinical practices. The PBRNs are characterized by an organizational framework that transcends a single site or study and serves as the clinical research "laboratory" for conducting comparative-effectiveness studies using patient-oriented measures. The PBRN approach to research has many benefits, including enhanced generalizability of results, pooling of resources, rapid patient recruitment, and collaborative opportunities. However, multisite research also brings challenges related to the protection of human participants and institutional review board oversight. CLINICAL AND RESEARCH ADVANTAGES: Athletic training studies frequently include relatively few participants and, consequently, are able to detect only large effects. The incidence of injury at a single site is sufficiently low that gathering enough data to adequately power a treatment study may take many years. Collaborative efforts across diverse clinical practice environments can yield larger patient samples to overcome the limitations inherent in single-site research efforts.

Behavioral health integration: an essential element of population-based healthcare redesign

The fundamental aim of healthcare reform is twofold: to provide health insurance coverage for most of the citizens currently uninsured, thereby granting them access to healthcare; and to redesign the overall healthcare system to provide better care and achieve the triple aim (better health for the population, better healthcare for individuals, and at less cost). The foundation for this improved system will rest on a redesigned (i.e., sufficiently comprehensive and integrated) system of primary care, with which all other providers, services, and sites of care are associated. The Patient-Centered Medical Home (PCMH) and its congeners are the best current examples of the kind of primary care that can achieve the triple aim, if they can become sufficiently comprehensive and can adequately integrate services. This means fully integrating behavioral healthcare into the PCMH, a difficult task under the most favorable circumstances. Creating functioning accountable care organizations is an even more daunting task: this requires new principles of collaborating and financing and the current prototypes have generally failed to incorporate behavioral healthcare sufficient to meet even the basic needs of the target population. This paper will discuss (1) the case for and the difficulties associated with integrating behavioral healthcare into primary care at three levels: the practice, the state, and the nation; and (2) how this looks clinically, operationally, and financially.

A novel protocol for streamlined IRB review of Practice-based Research Network (PBRN) card studies

PURPOSE

The "card study," in which clinicians record brief information about patient visits during usual clinical care, has long been a rapid method for conducting descriptive studies in practice-based research networks. Because an increasingly stringent regulatory environment has made conducting card studies difficult, we developed a streamlined method for obtaining card study institutional review board (IRB) approval.

METHODS

We developed a protocol for a study of the card study method, allowing new card study proposals of specific research questions to be submitted as addenda to the approved Card Study Protocol.

RESULTS

Seven card studies were proposed and approved under the Card Study Protocol during the first year after implementation, contrasted with one-card study proposed in the previous year. New card study ideas submitted as addenda to an approved protocol appeared to increase IRB comfort with the card study as a minimal risk method while reducing the hurdles to developing new study ideas.

CONCLUSIONS

A Card Study Protocol allowing new study questions to be submitted as addenda decreases time between idea generation and IRB approval. Shortened turn-around times may be useful for translating ideas into action while reducing regulatory burden.

Card studies for observational research in practice

PURPOSE

Observational studies that collect patient-level survey data at the point-of-care are often called card studies. Card studies have been used to describe clinical problems, management, and outcomes in primary care for more than 30 years. In this article we describe 2 types of card studies and the methods for conducting them.

METHODS

We undertook a descriptive review of card studies conducted in 3 Colorado practice-based research networks and several other networks throughout the United States. We summarized experiences of the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP).

RESULTS

Card studies can be designed to study specific conditions or care (clinicians complete a card when they encounter patients who meet inclusion criteria) and to determine trends and prevalence of conditions (clinicians complete a card on all patients seen during a period). Data can be collected from clinicians and patients and can be linked.

CONCLUSIONS

Card studies provide cross-sectional descriptive data about clinical care, knowledge and behavior, perception of care, and prevalence of conditions. Card studies remain a robust method for describing primary care.

Mining the mind research network: a novel framework for exploring large scale, heterogeneous translational neuroscience research data sources

A neuroinformatics (NI) system is critical to brain imaging research in order to shorten the time between study conception and results. Such a NI system is required to scale well when large numbers of subjects are studied. Further, when multiple sites participate in research projects organizational issues become increasingly difficult. Optimized NI applications mitigate these problems. Additionally, NI software enables coordination across multiple studies, leveraging advantages potentially leading to exponential research discoveries. The web-based, Mind Research Network (MRN), database system has been designed and improved through our experience with 200 research studies and 250 researchers from seven different institutions. The MRN tools permit the collection, management, reporting and efficient use of large scale, heterogeneous data sources, e.g., multiple institutions, multiple principal investigators, multiple research programs and studies, and multimodal acquisitions. We have collected and analyzed data sets on thousands of research participants and have set up a framework to automatically analyze the data, thereby making efficient, practical data mining of this vast resource possible. This paper presents a comprehensive framework for capturing and analyzing heterogeneous neuroscience research data sources that has been fully optimized for end-users to perform novel data mining.

Handheld vs. laptop computers for electronic data collection in clinical research: a crossover randomized trial

OBJECTIVE

To compare users' speed, number of entry errors and satisfaction in using two current devices for electronic data collection in clinical research: handheld and laptop computers.

DESIGN

The authors performed a randomized cross-over trial using 160 different paper-based questionnaires and representing altogether 45,440 variables. Four data coders were instructed to record, according to a random predefined and equally balanced sequence, the content of these questionnaires either on a laptop or on a handheld computer. Instructions on the kind of device to be used were provided to data-coders in individual sealed and opaque envelopes. Study conditions were controlled and the data entry process performed in a quiet environment.

MEASUREMENTS

The authors compared the duration of the data recording process, the number of errors and users' satisfaction with the two devices. The authors divided errors into two separate categories, typing and missing data errors. The original paper-based questionnaire was used as a gold-standard.

RESULTS

The overall duration of the recording process was significantly reduced (2.0 versus 3.3 min) when data were recorded on the laptop computer (p < 0.001). Data accuracy also improved. There were 5.8 typing errors per 1,000 entries with the laptop compared to 8.4 per 1,000 with the handheld computer (p < 0.001). The difference was even more important for missing data which decreased from 22.8 to 2.9 per 1,000 entries when a laptop was used (p < 0.001). Users found the laptop easier, faster and more satisfying to use than the handheld computer.

CONCLUSIONS

Despite the increasing use of handheld computers for electronic data collection in clinical research, these devices should be used with caution. They double the duration of the data entry process and significantly increase the risk of typing errors and missing data. This may become a particularly crucial issue in studies where these devices are provided to patients or healthcare workers, unfamiliar with computer technologies, for self-reporting or research data collection processes.

The Perfusion Downunder collaborative database project

The Perfusion Downunder Collaboration provides research infrastructure and support to the Australian and New Zealand perfusion community, with the objective of determining best practices and producing relevant research publications. The Perfusion Downunder Collaborative Database (PDUCD) has been created for the purpose of collecting a dataset for cardiopulmonary bypass (CPB) procedures that includes integration with commercially available CPB data collection software. Initial testing of the PDUCD involved collection of data from four Australian and New Zealand hospitals from March to July 2007. Data from 513 procedures were compared with the concurrent Australasian Society of Cardiac and Thoracic Surgeons (ASCTS) database report to assess the validity of the collected data. Demographic, preoperative, and procedural variables were comparable between databases. Perfusion variables showed a median nasopharyngeal temperature of 36.7degrees C at separation from CPB (range, 35.3-37.5 degrees C), which was similar to maximum nasopharyngeal temperature (median, 36.8 degrees C). Median arterial flow and mean arterial pressure were 4.2 L/min and 57.2 mmHg, respectively. Control charts indicate a central tendency of 12.5 minutes for mean arterial pressure < 50 mmHg and 3.5 minutes for arterial flow < 1.6 L/min/m2 (cumulative time). There was no difference in median minimum and maximum blood glucose between diabetic and nondiabetic patients during CPB with 40% of patients receiving insulin. Median minimum and maximum activated clotting time (ACT) during CPB was 581 and 692 seconds, respectively. Outcome data for isolated coronary artery bypass grafting were similar for mortality (only) (both 1.8%). Initial data collection showed concurrent validity compared with the ASCTS database. The inclusion of a large quantity of calculated CPB variables in the dataset highlights the benefits of electronic data collection as a research tool within a collaborative research network and the potential for the evaluation of the relationships between patient risk factors, perfusion practice, and patient outcomes.

Data collection outcomes comparing paper forms with PDA forms in an office-based patient survey

PURPOSE

We compared the completeness of data collection using paper forms and using electronic forms loaded on handheld computers in an office-based patient interview survey conducted within the American Academy of Family Physicians National Research Network.

METHODS

We asked 19 medical assistants and nurses in family practices to administer a survey about pneumococcal immunizations to 60 older adults each, 30 using paper forms and 30 using electronic forms on handheld computers. By random assignment, the interviewers used either the paper or electronic form first. Using multilevel analyses adjusted for patient characteristics and clustering of forms by practice, we analyzed the completeness of the data.

RESULTS

A total of 1,003 of the expected 1,140 forms were returned to the data center. The overall return rate was better for paper forms (537 of 570, 94%) than for electronic forms (466 of 570, 82%) because of technical difficulties experienced with electronic data collection and stolen or lost handheld computers. Errors of omission on the returned forms, however, were more common using paper forms. Of the returned forms, only 3% of those gathered electronically had errors of omission, compared with 35% of those gathered on paper. Similarly, only 0.04% of total survey items were missing on the electronic forms, compared with 3.5% of the survey items using paper forms.

CONCLUSIONS

Although handheld computers produced more complete data than the paper method for the returned forms, they were not superior because of the large amount of missing data due to technical difficulties with the hand-held computers or loss or theft. Other hardware solutions, such as tablet computers or cell phones linked via a wireless network directly to a Web site, may be better electronic solutions for the future.

Automated inter-rater reliability assessment and electronic data collection in a multi-center breast cancer study

Background

The choice between paper data collection methods and electronic data collection (EDC) methods has become a key question for clinical researchers. There remains a need to examine potential benefits, efficiencies, and innovations associated with an EDC system in a multi-center medical record review study.

Methods

A computer-based automated menu-driven system with 658 data fields was developed for a cohort study of women aged 65 years or older, diagnosed with invasive histologically confirmed primary breast cancer (N = 1859), at 6 Cancer Research Network sites. Medical record review with direct data entry into the EDC system was implemented. An inter-rater and intra-rater reliability (IRR) system was developed using a modified version of the EDC.

Results

Automation of EDC accelerated the flow of study information and resulted in an efficient data collection process. Data collection time was reduced by approximately four months compared to the project schedule and funded time available for manuscript preparation increased by 12 months. In addition, an innovative modified version of the EDC permitted an automated evaluation of inter-rater and intra-rater reliability across six data collection sites.

Conclusion

Automated EDC is a powerful tool for research efficiency and innovation, especially when multiple data collection sites are involved.

Using digital videos displayed on personal digital assistants (PDAs) to enhance patient education in clinical settings

OBJECTIVES

To evaluate the effects of using an audiovisual animation (i.e., digital video) displayed on a personal digital assistant (PDA) for patient education in a clinical setting.

METHODS

Quasi-experimental study of a prospective technology intervention conducted in an outpatient infectious diseases clinic at an academic medical center. Subjects responded to questions immediately before, immediately after, and 4-6 weeks after watching a digital video on a PDA. Outcome measures include participant knowledge of disease, knowledge of medications, and knowledge of adherence behaviors; attitudes toward the video and PDA; self-reported adherence; and practicality of the intervention.

RESULTS

Fifty-one English-speaking adults who were initiating or taking medications for the treatment of HIV/AIDS participated in the study. At visit one, statistically significant improvements in knowledge of disease (p<0.005; paired t-test), knowledge of medications (p<0.005; paired t-test), and knowledge of adherence behaviors (p<0.05; ANOVA) were measured after participants watched the PDA-based video. At visit two (4-6 weeks later), statistically significant improvements in self-reported adherence to the medication regimens (p<0.005; paired t-test) were reported. Participants liked the PDA-based video and indicated that it was an appropriate medium for learning, regardless of their baseline literacy skills. The video education process was estimated to take 25 min of participant time and was viewed in both private and semi-private locations.

CONCLUSIONS

Technology-assisted education using a digital video delivered via PDA is a convenient and potentially powerful way to deliver health messages. The intervention was implemented efficiently with participants of a variety of ages and educational levels, and in a range of locations within clinical environments. Additional study of this methodology is warranted.

Practice-based research network studies in the age of HIPAA

PURPOSE

We wanted to explore potential effects of the Health Insurance Portability and Accountability Act (HIPAA) on research activities of practice-based research networks (PBRNs).

METHODS

To understand the approaches PBRNs are using to advance their research while adhering to HIPAA standards, we combined a literature review, our experiences, and discussions with local HIPAA officers, PBRN researchers in the United States, and individuals involved in drafting HIPAA.

RESULTS

HIPAA requires researchers to pay special attention to how they handle patients' protected health information (PHI). For researchers working within PBRNs, which collect information from patients and health care professionals in multiple institutions, the HIPAA Privacy Rule presents additional challenges. PBRN researchers can obtain patient authorization to use PHI, but this process is difficult and may taint the findings of some research studies. Some institutions may allow patients to provide a blanket authorization for study recruitment. PBRNs additionally can collect only "de-identified" data (data with identifying information removed) or, with a data use agreement, can work with a limited data set. PBRNs that blend quality improvement and research can work with PHI, but the researcher and practices must enter into a business agreement. PBRN researchers may need to play active, educational roles in institutional privacy boards to facilitate their research.

CONCLUSIONS

There are a number of ways for PBRN researchers to comply with HIPAA short of obtaining patient consent and authorization for every study. Careful planning and consideration of HIPAA issues during study design can go a long way toward reducing frustration later.

Infrastructure requirements for practice-based research networks

BACKGROUND

The practice-based research network (PBRN) is the basic laboratory for primary care research. Although most PBRNs include some common elements, their infrastructures vary widely. We offer suggestions for developing and supporting infrastructures to enhance PBRN research success.

METHODS

Information was compiled based on published articles, the PBRN Resource Center survey of 2003, our PBRN experiences, and discussions with directors and coordinators from other PBRNs.

RESULTS

PBRN research ranges from observational studies, through intervention studies, clinical trials, and quality of care research, to large-scale practice change interventions. Basic infrastructure elements such as a membership roster, a board, a director, a coordinator, a news-sharing function, a means of addressing requirements of institutional review boards and the Health Insurance Portability and Accountability Act, and a network meeting must exist to support these initiatives. Desirable elements such as support staff, electronic medical records, multiuser databases, mentoring and development programs, mock study sections, and research training are costly and difficult to sustain through project grant funds. These infrastructure elements must be selected, configured, and sized according to the PBRN's self-defined research mission. Annual infrastructure costs are estimated to range from $69,700 for a basic network to $287,600 for a moderately complex network.

CONCLUSIONS

Well-designed and properly supported PBRN infrastructures can support a wide range of research of great direct value to patients and society. Increased and more consistent infrastructure support could generate an explosion of pragmatic, generalizable knowledge about currently understudied populations, settings, and health care problems.

Infrastructure requirements for practice-based research networks

BACKGROUND

The practice-based research network (PBRN) is the basic laboratory for primary care research. Although most PBRNs include some common elements, their infrastructures vary widely. We offer suggestions for developing and supporting infrastructures to enhance PBRN research success.

METHODS

Information was compiled based on published articles, the PBRN Resource Center survey of 2003, our PBRN experiences, and discussions with directors and coordinators from other PBRNs.

RESULTS

PBRN research ranges from observational studies, through intervention studies, clinical trials, and quality of care research, to large-scale practice change interventions. Basic infrastructure elements such as a membership roster, a board, a director, a coordinator, a news-sharing function, a means of addressing requirements of institutional review boards and the Health Insurance Portability and Accountability Act, and a network meeting must exist to support these initiatives. Desirable elements such as support staff, electronic medical records, multiuser databases, mentoring and development programs, mock study sections, and research training are costly and difficult to sustain through project grant funds. These infrastructure elements must be selected, configured, and sized according to the PBRN's self-defined research mission. Annual infrastructure costs are estimated to range from $69,700 for a basic network to $287,600 for a moderately complex network.

CONCLUSIONS

Well-designed and properly supported PBRN infrastructures can support a wide range of research of great direct value to patients and society. Increased and more consistent infrastructure support could generate an explosion of pragmatic, generalizable knowledge about currently understudied populations, settings, and health care problems.