The cost of operating institutional review boards (IRBs)

Caracterización de un grupo de comités de ética en investigación en Colombia

En las últimas décadas, los ensayos clínicos controlados patrocinados por la industria farmacéutica han aumentado de manera considerable, lo cual ha llevado a que sea necesario un mayor control y acompañamiento por parte de las entidades reguladoras, y de los comités de ética, para garantizar el adecuado cumplimiento de las normas éticas establecidas y de las buenas prácticas clínicas en general. En Colombia, el Instituto Nacional de Vigilancia de Medicamentos y Alimentos (Invima), la entidad reguladora en el país, reglamenta y vigila el funcionamiento de la investigación clínica con medicamentos. En el 2008, esta entidad emitió la Resolución 2378, normatividad bajo la cual se establecen y regulan los diferentes actores de la investigación en Colombia, incluidos los comités de ética. Después de varios años de vigencia de esta norma, es necesario conocer si los comités de ética en investigación en Colombia funcionan de acuerdo con esta normativa, y determinar el estado de la implementación de lo solicitado en la regulación. Con este objetivo se diseñó una encuesta para ser respondida en forma voluntaria y se obtuvo respuesta de 25 de los 69 comités certificados en Colombia. El análisis fue posible en 22 de ellos cuya información estaba completa. En comparación con estudios anteriores, se observaron cambios favorables en el desarrollo y la organización de acuerdo con la norma actual propuesta.

IRB Process Improvements: A Machine Learning Analysis

OBJECTIVE

Clinical research involving humans is critically important, but it is a lengthy and expensive process. Most studies require institutional review board (IRB) approval. Our objective is to identify predictors of delays or accelerations in the IRB review process and apply this knowledge to inform process change in an effort to improve IRB efficiency, transparency, consistency and communication.

METHODS

We analyzed timelines of protocol submissions to determine protocol or IRB characteristics associated with different processing times. Our evaluation included single variable analysis to identify significant predictors of IRB processing time and machine learning methods to predict processing times through the IRB review system. Based on initial identified predictors, changes to IRB workflow and staffing procedures were instituted and we repeated our analysis.

RESULTS

Our analysis identified several predictors of delays in the IRB review process including type of IRB review to be conducted, whether a protocol falls under Veteran's Administration purview and specific staff in charge of a protocol's review.

CONCLUSIONS

We have identified several predictors of delays in IRB protocol review processing times using statistical and machine learning methods. Application of this knowledge to process improvement efforts in two IRBs has led to increased efficiency in protocol review. The workflow and system enhancements that are being made support our four-part goal of improving IRB efficiency, consistency, transparency, and communication.

Time required to review research protocols at 10 Veterans Affairs Institutional Review Boards

BACKGROUND

Despite perceptions that institutional review boards (IRBs) delay research, little is known about how long it takes to secure IRB approval. We retrospectively quantified IRB review times at 10 large Veterans Affairs (VA) IRBs.

METHODS

We collected IRB records pertaining to a stratified random sample of research protocols drawn from 10 of the 26 largest VA IRBs. Two independent analysts abstracted dates from the IRB records, from which we calculated overall and incremental review times. We used multivariable linear regression to assess variation in total and incremental review times by IRB and review level (i.e., exempt, expedited, or full board) and to identify potential targets for efforts to improve the efficiency and uniformity of the IRB review process.

RESULTS

In a sample of 277 protocols, the mean review time was 112 d (95% confidence interval [CI]: 105-120). Compared with full-board reviews at IRB 1, average review times at IRBs 3, 8, 9, and 10 were 27 (95% CI: 6-48), 37 (95% CI: 11-63), 45 (95% CI: 20-69), and 24 (95% CI: 2-45) d shorter, and at IRB 6, times were 56 (95% CI: 28-84) d longer. Across all IRBs, expedited reviews were 44 (95% CI: 30-58) d shorter on average than were full-board reviews, with no significant difference between exempt and full-board reviews. However, after subtracting the time required for Research and Development Committee review, exempt reviews were 21 (95% CI: 1-41) d shorter on average than were full-board reviews.

CONCLUSIONS

IRB review times differ significantly by IRB and review level. Few VA IRBs approach a consensus panel goal of 60 d for IRB review. The unexpectedly longer review times for exempt protocols in the VA can be attributed to time required for Research and Development Committee review. Prospective, routine collection of key time points in the IRB review process could inform IRB-specific initiatives for reducing VA IRB review times.

Prioritizing Initiatives for Institutional Review Board (IRB) Quality Improvement

Background

Institutional Review Boards (IRBs) have been criticized for inconsistency, delay, and bias, suggesting an opportunity for quality improvement. To aid such quality improvement, this study aimed at determining IRB members' and investigators' priorities regarding IRB review at 10 Veterans Affairs (VA) IRBs.

Methods

680 anonymous internet surveys were sent to 252 IRB members and staff, and 428 principal investigators and project coordinators at 9 VA Medical Centers and the VA Central IRB. Surveys included 27 statements adapted from Koocher and Kieth-Spiegel's IRB-RAT describing IRB activities or functions (e.g., "An IRB that is open to reversing its earlier decisions"). Respondents indicated how each statement described both their "ideal" and "actual" IRBs. The difference between the ratings of the actual and ideal IRBs was calculated for each item along with estimated 95% confidence intervals. Ratings outside those intervals indicated activities or functions with relatively good or poor performance compared to the ideal IRB.

Results

390 (57.4%) responses from 165 IRB members and staff (65.5%) and 225 investigators and project coordinators (52.6%) demonstrated that these IRBs were closest to the ideal when protecting human subjects, treating investigators with respect, and taking appropriate action for alleged scientific misconduct. The IRBs were furthest from the ideal regarding duplicative forms, timeliness of review, and provision of complete rationales for decisions. Although IRB members reported near-ideal willingness to reverse earlier decisions, investigators rated this capacity far from ideal. Investigators rated IRB members' knowledge about procedures and policy as nearly ideal, but IRB members themselves rated this aspect far from ideal. Noteworthy site-level differences in the ratings of IRB functions and activities were also identified.

Conclusions

Although these VA IRBs perform well in some areas, these data support the need for ongoing quality improvement. The described method of administering and analyzing the IRB-RAT may help identify and monitor site- and activity- specific initiatives for quality improvement.

Using the IRB Researcher Assessment Tool to Guide Quality Improvement

Institutional Review Boards (IRBs) are intended to protect those who participate in research. However, because there is no established measure of IRB quality, it is unclear whether these committees achieve their goal. The IRB Researcher Assessment Tool is a previously validated, internally normed, proxy measure of IRB quality that assesses 45 distinct IRB activities and functions. We administered this instrument to a sample of investigators and IRB members at a large urban VA Medical Center. We describe a systematic approach to analyze and interpret survey responses that can identify the IRB activities and functions most in need of quality improvement. The proposed approach to empirical data analysis and presentation could inform local initiatives to improve the quality of IRB review.

Time required for institutional review board review at one Veterans Affairs medical center

IMPORTANCE

Despite growing concern that institutional review boards (IRBs) impose burdensome delays on research, little is known about the time required for IRB review across different types of research.

OBJECTIVE

To measure the overall and incremental process times for IRB review as a process of quality improvement.

DESIGN, SETTING, AND PARTICIPANTS

After developing a detailed process flowchart of the IRB review process, 2 analysts abstracted temporal data from the records pertaining to all 103 protocols newly submitted to the IRB at a large urban Veterans Affairs medical center from June 1, 2009, through May 31, 2011. Disagreements were reviewed with the principal investigator to reach consensus. We then compared the review times across review types using analysis of variance and post hoc Scheffé tests after achieving normally distributed data through logarithmic transformation.

MAIN OUTCOMES AND MEASURES

Calendar days from initial submission to final approval of research protocols.

RESULTS

Initial IRB review took 2 to 4 months, with expedited and exempt reviews requiring less time (median [range], 85 [23-631] and 82 [16-437] days, respectively) than full board reviews (median [range], 131 [64-296] days; P = .008). The median time required for credentialing of investigators was 1 day (range, 0-74 days), and review by the research and development committee took a median of 15 days (range, 0-184 days). There were no significant differences in credentialing or research and development times across review types (exempt, expedited, or full board). Of the extreme delays in IRB review, 80.0% were due to investigators' slow responses to requested changes. There were no systematic delays attributable to the information security officer, privacy officer, or IRB chair.

CONCLUSIONS AND RELEVANCE

Measuring and analyzing review times is a critical first step in establishing a culture and process of continuous quality improvement among IRBs that govern research programs. The review times observed at this IRB are substantially longer than the 60-day target recommended by expert panels. The method described here could be applied to other IRBs to begin identifying and improving inefficiencies.

Views of IRBs Concerning their Local Ecologies: Perceptions of Relationships, Systems, and Tensions between IRBs and their Institutions

BACKGROUND

Research has generally examined institutional review boards (IRBs) in isolation, but critical questions arise of how these entities fit into the larger institutional contexts in which they operate and what the implications may be.

METHODS

Semi-structured interviews were conducted with leaders of IRBs from among the top 240 institutions receiving funding from the National Institutes of Health.

RESULTS

Interviewees felt that institutions may affect IRBs through both broad, indirect features (e.g., size, type of research, and culture of the institution), and more direct, IRB-related factors (e.g., amount of leadership and resource support for the IRB). Interviewees thought that institutional support of IRBs ranged from financial to non-financial, direct and indirect, and that these institutional factors can mold amounts of IRB staff and education, audits, and education of principal investigators (PIs), and tensions IRBs had to address. Respondents felt that these factors can in turn potentially affect IRB reviews of protocols and interactions with principle investigators (PIs). Within the complex systems of an institution, IRBs felt that PIs' experiences and complaints about the IRB to institutional leaders may also shape how the institution related to the IRB.

CONCLUSIONS

These data are the first to show how IRBs perceive themselves as working within the contexts of dynamic local institutional relationships and systems that pose challenges and tensions that can potentially affect critical aspects of IRB functioning. The findings have implications for practice, future research, and policy.

U. S. Health Researchers Review their Ethics Review Boards: A Qualitative Study

VIRTUALLY ALL RESEARCH INVOLVING HUMAN subjects in the United States must be reviewed by an institutional review board, a form of research ethics review board. This article reports the results of qualitative research on how investigators regard this regulatory regime. Interviews were conducted with forty investigators conducting health-related research. Most respondents shared the regulations' goals, but doubted that the regulations, as implemented, promoted these goals efficiently, effectively and fairly. The interviews suggest that efforts to raise researchers' ethical consciousness have been, over time, quite successful, but that implementation of the regulations remains problematic. Research aimed at better defining the problem to be solved b y the r egulatory sy stem, and at a ssessing the effectiveness of the regulatory tools for solving properly defined problems, could guide a more productive debate about human subject protection.

Financial Burden of Emergency Preparedness on an Urban, Academic Hospital

This study assessed the direct human resource costs of a hospital's emergency preparedness planning (in 2005) by surveying participants retrospectively. Forty participants (74% of the identified population) were surveyed. Using the self-reported hourly salary of the participant, a direct salary cost was calculated for each participant. The population was 40% male and 60% female; 65% had a graduate degree or higher; 65% were administrators; 35% were clinicians; and 50% reported that their job description included a reference to emergency planning activities. All participants spent a combined total of 3,654.25 hours on emergency preparedness activities, including 20.1% on personal education/train-ing; 11.6% on educating other people; 39.3% on paperwork or equipment maintenance; 22.2% on attendance at meetings; 5.6% on drill participation; and <1% on other activities. Considering the participants' hourly salary, direct personal costs spent on emergency preparedness activities at the institution totaled US$232,417. Ten percent, all of whom were physicians, reported no compen-sation for their emergency preparedness efforts at the hospital level.

As much as these results illustrate the strong commitment of the institu-tion to its community, they represent a heavy burden in light of the oftenunfunded mandate of emergency preparedness planning that a hospital may incur. Such responsibility is carried to some extent by all hospitals.

Burdens on research imposed by institutional review boards: the state of the evidence and its implications for regulatory reform

CONTEXT

Federal regulations mandate independent review and approval by an "institutional review board" (IRB) before studies that involve human research subjects may begin. Although many researchers strongly support the need for IRB review, they also contend that it is burdensome when it imposes costs that do not add to the protections afforded to research participants and that this burden threatens the viability of research. The U.S. Department of Health and Human Services recently announced its intention to reform the regulations governing IRB review.

METHODS

We used a search of the PubMed database, supplemented by a bibliographic review, to identify all existing primary data on the costs of IRB review. "Costs" were broadly defined to include both expenditures of time or money and constraints imposed on the scope of the research. Burdensome costs were limited to those that did not contribute to greater protections for the participants.

FINDINGS

Evidence from a total of fifty-two studies shows that IRBs operate at different levels of efficiency; that waiting to obtain IRB approval has, in some instances, delayed project initiation; that IRBs presented with identical protocols sometimes asked for different and even competing revisions; and that some decisions made (and positions held) by IRBs are not in accord with federal policy guidance.

CONCLUSIONS

While the evidence is sufficient to conclude that there is burden associated with IRB review, it is too limited to allow for valid estimates of its magnitude or to serve as the basis for formulating policies on IRB reform. The single exception is multicenter research, for which we found that review by several local IRBs is likely to be burdensome. No mechanism currently exists at the national level to gather systematic evidence on the intersection between research and IRB review. This gap is of concern in light of the changing nature of research and the increasingly important role that research is envisioned to play in improving the overall quality of health care.

A systematic review of the empirical literature evaluating IRBs: what we know and what we still need to learn

Institutional review boards (IRBs) are integral to the U.S. system of protection of human research participants. Evaluation of IRBs, although difficult, is essential. To date, no systematic review of IRB studies has been published. We conducted a systematic review of empirical studies of U.S. IRBs to determine what is known about the function of IRBs and to identify gaps in knowledge. A structured search in PubMed identified forty-three empirical studies evaluating U.S. IRBs. Studies were included if they reported an empirical investigation of the structure, process, outcomes, effectiveness, or variation of U.S. IRBs. The authors reviewed each study to extract information about study objectives, sample and methods, study results, and conclusions. Empirical evidence collected in forty-three published studies shows that for review of a wide range of types of research, U.S. IRBs differ in their application of the federal regulations, in the time they take to review studies, and in the decisions made. Existing studies show evidence of variation in multicenter review, inconsistent or ambiguous interpretation of the federal regulations, and inefficiencies in review. Despite recognition of a need to evaluate effectiveness of IRB review, no identified published study included an evaluation of IRB effectiveness. Multiple studies evaluating the structure, process, and outcome of IRB review in the United States have documented inconsistencies and inefficiencies. Efforts should be made to address these concerns. Additional research is needed to understand how IRBs accomplish their objectives, what issues they find important, what quality IRB review is, and how effective IRBs are at protecting human research participants.

Use of health information technology to advance evidence-based care: lessons from the VA QUERI program

BACKGROUND

As the Department of Veterans Affairs (VA) Health Services Research and Development Service's Quality Enhancement Research Initiative (QUERI) has progressed, health information technology (HIT) has occupied a crucial role in implementation research projects.

OBJECTIVES

We evaluated the role of HIT in VA QUERI implementation research, including HIT use and development, the contributions implementation research has made to HIT development, and HIT-related barriers and facilitators to implementation research.

PARTICIPANTS

Key informants from nine disease-specific QUERI Centers.

APPROACH

Documentation analysis of 86 implementation project abstracts followed up by semi-structured interviews with key informants from each of the nine QUERI centers. We used qualitative and descriptive analyses.

RESULTS

We found: (1) HIT provided data and information to facilitate implementation research, (2) implementation research helped to further HIT development in a variety of uses including the development of clinical decision support systems (23 of 86 implementation research projects), and (3) common HIT barriers to implementation research existed but could be overcome by collaborations with clinical and administrative leadership.

CONCLUSIONS

Our review of the implementation research progress in the VA revealed interdependency on an HIT infrastructure and research-based development. Collaboration with multiple stakeholders is a key factor in successful use and development of HIT in implementation research efforts and in advancing evidence-based practice.

Improving Participant Comprehension in the Informed Consent Process

PURPOSE

To critically analyze studies published within the past decade about participants' comprehension of informed consent in clinical research and to identify promising intervention strategies.

DESIGN

Integrative review of literature.

METHODS

The Cumulative Index of Nursing and Allied Health Literature (CINAHL), PubMed, and the Cochrane Database of Systematic Reviews and Cochrane Central Register of Controlled Trials were searched. Inclusion criteria included studies (a) published between January 1, 1996 and January 1, 2007, (b) designed as descriptive or interventional studies of comprehension of informed consent for clinical research, (c) conducted in nonpsychiatric adult populations who were either patients or volunteer participants, (d) written in English, and (e) published in peer-reviewed journals.

FINDINGS

Of the 980 studies identified, 319 abstracts were screened, 154 studies were reviewed, and 23 met the inclusion criteria. Thirteen studies (57%) were descriptive, and 10 (43%) were interventional. Interventions tested included simplified written consent documents, multimedia approaches, and the use of a trained professional (consent educator) to assist in the consent process. Collectively, no single intervention strategy was consistently associated with improved comprehension. Studies also varied in regard to the definition of comprehension and the tools used to measure it.

CONCLUSIONS

Despite increasing regulatory scrutiny, deficiencies still exist in participant comprehension of the research in which they participate, as well as differences in how comprehension is measured and assessed. No single intervention was identified as consistently successful for improving participant comprehension, and results indicated that any successful consent process should at a minimum include various communication modes and is likely to require one-to-one interaction with someone knowledgeable about the study.

Viewpoint: IRBs, hospital ethics committees, and the need for "translational informed consent"

In this paper, the authors contrast the model of informed consent oversight employed by most IRBs with that most commonly employed by hospital ethics committees at academic medical centers. The challenge of developing adequate consent forms is complicated by competing aims that are difficult to reconcile. "Completeness" in conveying highly technical information is often at odds with comprehensibility for lay audiences. The authors argue that the problems posed by consent are tied to the fact that in clinical research, consent documents are perceived as the primary mechanism for securing informed consent, as opposed to viewing the consent process and conversation as the primary mechanism. Concerns for informed consent should be refocused to the consent process rather than IRB review of documents, using teams of consent monitors, and this new emphasis should be piloted through the National Institute of Health's recent turn toward clinical and translational science awards.

Reforming the Oversight of Multi-Site Clinical Research: A Review of Two Possible Solutions

The current system for the ethical oversight of clinical research suffers from structural, procedural, and performance assessment problems. Initially conceived primarily to handle local investigator-initiated single-site studies, the system of institutionally-based committee review has become progressively more inefficient given the increased prevalence of commercially or federally sponsored multi-center trials. To date, proposed solutions do not adequately address these problems. Beginning with a review of these structural, procedural, and performance assessment problems, this article will then consider two proposals for addressing these deficiencies: (a) regional ethics organizations; and (b) IRBNet, a newly developed web-based program for cooperative IRB review. The strengths and weaknesses of these two approaches will be evaluated in light of recent experience with centralized review. The proposal to establish a system of regional ethics organizations presents a comprehensive approach to many of the problems faced by the current system. However, IRBNet offers an immediate and feasible solution to many of the problems faced by the review of multi-site clinical studies.

Variability in the costs of institutional review board oversight

BACKGROUND

Previous studies have shown wide differences between institutions in economies of scale with regard to the costs of institutional review board (IRB) oversight of research. In this study, the authors explored variability among IRB costs, taking into account organizational size, components of the costs of oversight, and protocol type.

METHOD

The authors conducted a survey of academic medical centers to collect information on resource utilization associated with IRB oversight in 2002. They used national cost weights to assign a cost to each type of resource used, and summed weighted resource utilization for IRB costs. Descriptive statistics were generated for costs over all, tertile of protocol volume, cost component, and type of review. They also determined where the greatest cost variability is found.

RESULTS

IRB costs per protocol reviewed are highly variable both overall and within tertiles of volume. Higher-volume institutions have lower costs, which is indicative of economies of scale. However, not all components of IRB costs (e.g., board time) are subject to economies of scale. Expedited reviews of protocols are not less expensive at low-volume institutions.

CONCLUSIONS

IRB costs for oversight are highly variable, and only some of the variation may be attributable to economies of scale. Given such wide variation in costs, the authors conclude that some institutions are conducting reviews in a manner that is inefficient or of low quality. Future work is needed to determine specific practices in reviews, and what leads to the best quality and most efficient oversight and review system.

New challenges facing interinstitutional social science and educational program evaluation research at academic health centers: a case study from the ELAM program

Since the mid-1990s, the protection of human subjects through institutional review boards (IRBs) has progressively broadened in scope. In this case study, the authors describe their challenges in effectively handling IRB processes to conduct educational and social sciences research within academic health centers, particularly (1) complications in conducting longitudinal interinstitutional research that involves multiple IRBs, each with different procedures that changed over ten years; and (2) factors affecting consent form and survey response rates when applying the biomedical IRB process to obtain the consent of human subjects for participation in social and educational research. The authors had a unique opportunity to follow the effect of changes in consent forms (from no form to a one-page form to a three-page form requiring signature of a witness), ways of administration (in person or by mail), and time of administration (at the time of the program or years later) on consent form and survey response rates among medical and dental school faculty members. The authors explore the extended timelines required for data collection and increased costs in dealing with these issues, as well as the effects on response rates of consent form language and administration procedures. The authors recommend strategies to lessen adverse effects of dealing with multiple IRBs at different institutions for social science and educational research, and discuss policy implications for funders, institutions and investigators.

Economies of Scale in Institutional Review Boards

BACKGROUND

Research with human subjects is essential for most clinical and social science research. As such, the ethical treatment of subjects, including the role of Institutional Review Boards (IRBs), is of paramount concern. The prevailing system of IRBs in the United States reflects an integrated approach in which research organizations have their own local IRB. Recent regulatory changes and a few high-profile problems have prompted proposals for greater investments in IRBs.

OBJECTIVES

We conducted regression analyses, looking at how IRB size was associated with IRB costs (economies of scale).

RESEARCH DESIGN

We studied data from a cross-sectional survey.

SUBJECTS

We studied IRBs at Veterans Affairs (VA) and VA-affiliated medical centers (n = 109); 81 (73%) IRB administrators completed the survey. Fourteen of the administrators had missing data and were excluded from final analysis, leaving a sample of 67.

MEASURES

The primary dependent variable was IRB costs in 2001, which we estimated from the survey. Independent variables included IRB size measured as the number of actions (ie, number of initial reviews, amendments, continuing/annual reviews, and harms/adverse event reports) reviewed by the IRB in the last year.

RESULTS

The results indicate that very large economies of scale exist, especially for IRBs that handle fewer than 150 actions per year.

CONCLUSIONS

A discussion of the costs of benefits of having 3000 to 5000 local IRBs in the United States is warranted because other organizational arrangements could be economically and socially advantageous.

Applying the common rule to public health agencies: questions and tentative answers about a separate regulatory regime

No one questions the importance of protecting human subjects of research, but over the past few years dissatisfaction has surfaced with the manner in which the protection is conferred by the federal regulatory system referred to as “The Common Rule. ” Some of the criticism surfaces in print. Some bubbles out anecdotally in conversations among researchers, with complaints about the review process being virtually inevitable whenever the topic arises. Like those in other disciplines that differ more or less dramatically from the world of clinical medical research around which the Common Rule was conceptualized, public health researchers, particularly those working in official public health agencies, have experienced some tension and difficulty in adapting to the rules. The Centers For Disease Control and Prevention (CDC) has adopted guidelines that require Common Rule review for “research” but not “practice” activities.