"We measure what we can measure": Struggles in defining and evaluating institutional review board quality

Research ethics and collaborative research in health and social care: Analysis of UK research ethics policies, scoping review of the literature, and focus group study

Current research ethics frameworks were developed on the footprint of biomedical, experimental research and present several pitfalls when applied to non-experimental social sciences. This work explores how the normative principles underpinning policy and regulatory frameworks of research ethics and the related operational processes work in practice in the context of collaborative health and social care research. The work was organised in three phases. First, UK research ethics policy documents were analysed thematically, with themes further organised under the categories of 'Principles' and 'Processes'. Next, we conducted a scoping review of articles about research ethics in the context of collaborative health and social care research, published in English between 2010 and 2022. We then held an exploratory focus group with ten academic researchers with relevant experience to gather their views on how the research ethics system works in practice in England (UK). The thematic framework developed in the first phase supported the analysis of the articles included in the scoping review and of focus group data. The analysis of policy documents identified twelve themes. All were associated to both a principle and a related operational process. The scoping review identified 31 articles. Across these, some themes were barely acknowledged (e.g., Compliance with legislation). Other themes were extensively covered (e.g., The working of Research Ethics Committees), often to discuss issues and limitations in how, in practice, the research ethics system and its processes deal with collaborative research and to suggest options for improvement. Focus group data were largely consistent with the findings of the scoping review. This work provides evidence of the poor alignment between how the research ethics system is normatively expected to work and how it works in practice and offers options that could make research ethics more fit for purpose when addressing collaborative research in health and social care.

How Do Accredited Organizations Evaluate the Quality and Effectiveness of Their Human Research Protection Programs?

BACKGROUND

Meaningfully evaluating the quality of institutional review boards (IRBs) and human research protection programs (HRPPs) is a long-recognized challenge. To be accredited by the Association for the Accreditation of Human Research Protection Programs (AAHRPP), organizations must demonstrate that they measure and improve HRPP "quality, effectiveness, and efficiency" (QEE). We sought to learn how AAHRPP-accredited organizations interpret and satisfy this standard, in order to assess strengths, weaknesses, and gaps in current approaches and to inform recommendations for improvement.

METHODS

We conducted 3 small-group interviews with a total of 19 participant representatives of accredited organizations at the 2019 AAHRPP annual meeting. Participants were eligible if they had familiarity with their organization's approach to satisfying the relevant QEE standard.

RESULTS

Participants reported lacking clear definitions for HRPP quality or effectiveness but described various approaches to assessing QEE, typically focused on turnaround time, compliance, and researcher satisfaction. Evaluation of IRB members was described as relatively superficial and information regarding research subject experience was not reported as central to QEE assessment, although participants described several efforts to improve consideration of patient, subject, and community perspectives in IRB review. Participants also described efforts to educate and build relationships with key stakeholders as important features of a high-quality HRPP. While generally satisfied with their approaches, participants expressed concern about resource and time constraints that pushed them to be reactive and automatic about QEE, rather than proactive and critical.

CONCLUSIONS

The relevant AAHRPP accreditation standard may obscure critical gaps in defining and measuring QEE elements. We recommend that AAHRPP: (1) offer a definition of QEE or require accredited organizations to provide their own, to help clarify the rationale and goals behind assessment and improvement efforts, and (2) require accredited organizations to establish QEE objectives and measures focused on participant outcomes and deliberative quality during protocol review.

Research ethics in a changing social sciences landscape

The role of research ethics committees, and research ethics issues more broadly are often not viewed in the context of the development of scientific methods and the academic community. This topic piece seeks to redress this gap. I begin with a brief outline of the changes we experience within the social sciences before exploring in more detail their impact on research ethics and the practices of research ethics committees. I conclude with recommendations for how the existing research ethics processes may be made more future-proof.

Engaging key stakeholders to overcome barriers to studying the quality of research ethics oversight

The primary purpose of Institutional Review Boards (IRBs) is to protect the rights and welfare of human research participants. Evaluation and measurement of how IRBs satisfy this purpose and other important goals are open questions that demand empirical research. Research on IRBs, and the Human Research Protection Programs (HRPPs) of which they are often a part, is necessary to inform evidence-based practices, policies, and approaches to quality improvement in human research protections. However, to date, HRPP and IRB engagement in empirical research about their own activities and performance has been limited. To promote engagement of HRPPs and IRBs in self-reflective research on HRPP and IRB quality and effectiveness, barriers to their participation need to be addressed. These include: extensive workloads, limited information technology systems, and few universally accepted or consistently measured metrics for HRPP/IRB quality and effectiveness. Additionally, institutional leaders may have concerns about confidentiality. Professional norms around the value of participating in this type of research are lacking. Lastly, obtaining external funding for research on IRBs and HRPPs is challenging. As a group of HRPP professionals and researchers actively involved in a research consortium focused on IRB quality and effectiveness, we identify potential levers for supporting and encouraging HRPP and IRB engagement in research on quality and effectiveness. We maintain that this research should be informed by the core principles of patient- and community-engaged research, in which members and key stakeholders of the community to be studied are included as key informants and members of the research team. This ensures that relevant questions are asked and that data are interpreted to produce meaningful recommendations. As such, we offer several ways to increase the participation of HRPP professionals in research as participants, as data sharers, and as investigators.