Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials

Developments in the Design, Conduct, and Reporting of Child Health Trials

To identify priority areas to improve the design, conduct, and reporting of pediatric clinical trials, the international expert network, Standards for Research (StaR) in Child Health, was assembled and published the first 6 Standards in Pediatrics in 2012. After a recent review summarizing the 247 publications by StaR Child Health authors that highlight research practices that add value and reduce research "waste," the current review assesses the progress in key child health trial methods areas: consent and recruitment, containing risk of bias, roles of data monitoring committees, appropriate sample size calculations, outcome selection and measurement, and age groups for pediatric trials. Although meaningful change has occurred within the child health research ecosystem, measurable progress is still disappointingly slow. In this context, we identify and review emerging trends that will advance the agenda of increased clinical usefulness of pediatric trials, including patient and public engagement, Bayesian statistical approaches, adaptive designs, and platform trials. We explore how implementation science approaches could be applied to effect measurable improvements in the design, conducted, and reporting of child health research.

Facilitators and Barriers to Pediatric Clinical Trial Recruitment and Retention in Rural and Community Settings: A Scoping Review of the Literature

Children in rural settings are underrepresented in clinical trials, potentially contributing to rural health disparities. We performed a scoping review describing available literature on barriers and facilitators impacting participation in pediatric clinical trials in rural and community-based (non-clinical) settings. Articles identified via PubMed, CINAHL, Embase, and Web of Science were independently double-screened at title/abstract and full-text levels to identify articles meeting eligibility criteria. Included articles reported on recruitment or retention activities for U.S.-based pediatric clinical studies conducted in rural or community-based settings and were published in English through January 2021. Twenty-seven articles describing 31 studies met inclusion criteria. Most articles reported on at least one study conducted in an urban or suburban or unspecified community setting (n=23 articles; 85%); fewer (n=10; 37%) reported on studies that spanned urban and rural settings or were set in rural areas. More studies discussed recruitment facilitators (n=25 studies; 81%) and barriers (n=19; 61%) versus retention facilitators (n=15; 48%) and barriers (n=8; 26%). Descriptions of recruitment and retention barriers and facilitators were primarily experiential or subjective. Recruitment and retention facilitators were similar across settings and included: contacts/reminders, community engagement and relationship-building, consideration of participant logistics, and incentives. Inadequate staff and resources were commonly cited recruitment and retention barriers. Few studies have rigorously examined optimal ways to recruit and retain rural participants in pediatric clinical trials. To expand the evidence base, future studies examining recruitment and retention strategies should systematically assess and report rurality and objectively compare relative impact of different strategies.

Knowledge Translation Strategies Used by Healthcare Professionals in Child Health Settings: An Updated Systematic Review

PROBLEM

Strategies assisting healthcare professionals to make evidence-based decisions are crucial for quality patient care and outcomes. To date, there is one systematic review (Albrecht et al., 2016) examining knowledge translation (KT) efforts in child health settings. This systematic review aims to provide an update on current evidence identifying KT interventions implementing research into child health settings.

ELIGIBILITY CRITERIA

Nine electronic databases were searched, restricted by date (2011-2018) and language (English). Eligibility included: 1) randomized controlled trials (RCT), controlled clinical trials (CCT), or controlled before-and-after (CBA) studies; 2) interventions implementing research into child health practice; and 3) outcomes were measured at the healthcare professional/process, patient, or economic level.

SAMPLE

Health care professionals working in child health settings.

RESULTS

48 studies (38 RCT, 7 CBA, 3 CCT) were included. Studies employed single (n = 34) and multiple (n = 14) interventions. The methodological quality of studies was moderate (n = 18), strong (n = 16) and weak (n = 14). Studies showing significant, positive effects included (n = 9) RCTs, (n = 3) CBAs and (n = 2) CCTs. These studies employed (n = 11) single KT interventions and (n = 3) multiple KT interventions. Interventions included educational (n = 6), reminders (n = 3), computerized decision supports (n = 2), multidisciplinary teams (n = 2) and financial and educational interventions combined (n = 1).

CONCLUSIONS

Effective KT strategies used by health care professionals in child health settings were found to be online education curriculums and computerized decision supports or reminders.

IMPLICATIONS

This review update serves as an up-to-date 'state of the science' on KT strategies used in pediatric health professionals' clinical practice, assessed by the most rigorous research designs.

Reporting of data monitoring committees and adverse events in paediatric trials: a descriptive analysis

OBJECTIVES

For 300 paediatric trials, we evaluated the reporting of: a data monitoring committee (DMC); interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints.

METHODS

For this cross-sectional evaluation, we randomly selected 300 paediatric trials published in 2012 from the Cochrane Central Register of Controlled Trials. We collected data on the reporting of a DMC; interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. We reported the findings descriptively and stratified by trial characteristics.

RESULTS

Eighty-five (28%) of the trials investigated drugs, and 18% (n=55/300) reported a DMC. The reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%, p<0.001) and industry-sponsored trials compared with those sponsored by other sources (n=16/50, 32% vs n=39/250, 16%, p=0.009). Trials that reported a DMC enrolled more participants than those that did not (median [range]): 224 (10-60480) vs 91 (10-9528) (p<0.001). Only 25% of these trials reported interim analyses, and 42% reported stopping rules. Less than half (n=143/300, 48%) of trials reported on adverse events, and 72% (n=215/300) reported on harm-related endpoints. Trials that reported a DMC compared with those that did not were more likely to report adverse events (n=43/55, 78% vs 100/245, 41%, p<0.001) and harm-related endpoints (n=52/55, 95% vs. 163/245, 67%, p<0.001). Only 32% of drug trials reported a DMC; 18% and 19% did not report on adverse events or harm-related endpoints, respectively.

CONCLUSIONS

The reporting of a DMC was infrequent, even among drug trials. Few trials reported stopping rules or interim analyses. Reporting of adverse events and harm-related endpoints was suboptimal.

Conducting clinical trials-costs, impacts, and the value of clinical trials networks: A scoping review

BACKGROUND

A significant barrier to conducting clinical trials is their high cost, which is driven primarily by the time and resources required to activate trials and reach accrual targets. The high cost of running trials has a substantial impact on their long-term feasibility and the type of clinical research undertaken.

METHODS

A scoping review of the empirical literature on the costs associated with conducting clinical trials was undertaken for the years 2001-2015. Five reference databases were consulted to elicit how trials costs are presented in the literature. A review instrument was developed to extract the content of in-scope papers. Findings were characterized by date and place of publication, clinical disease area, and network/cooperative group designation, when specified. Costs were captured and grouped by patient accrual and management, infrastructure, and the opportunity costs associated with industry funding for trials research. Cost impacts on translational research and health systems were also captured, as were recommendations to reduce trial expenditures. Since articles often cited multiple costs, multiple cost coding was used during data extraction to capture the range and frequency of costs.

RESULTS

A total of 288 empirical articles were included. The distribution of reported costs was: patient management and accrual costs (132 articles), infrastructure costs (118 articles) and the opportunity costs of industry sponsorship (72 articles). 221 articles reported on the impact of undertaking costly trials on translational research and health systems; of these, the most frequently reported consequences were to research integrity (52% of articles), research capacity (36% of articles) and running low-value trials (34% of articles). 254 articles provided recommendations to reduce trial costs; of these, the most frequently reported recommendations related to improvements in: operational efficiencies (33% of articles); patient accrual (24% of articles); funding for trials and transparency in trials reporting (18% of articles, each).

CONCLUSION

Key findings from the review are: 1) delayed trial activation has costs to budgets and research; 2) poor accrual leads to low-value trials and wasted resources; 3) the pharmaceutical industry can be a pragmatic, if problematic, partner in clinical research; 4) organizational know-how and successful research collaboration are benefits of network/cooperative groups; and 5) there are spillover benefits of clinical trials to healthcare systems, including better health outcomes, enhanced research capacity, and drug cost avoidance. There is a need for more economic evaluations of the benefits of clinical research, such as health system use (or avoidance) and health outcomes in cities and health authorities with institutions that conduct clinical research, to demonstrate the affordability of clinical trials, despite their high cost.

Advancing Randomized Controlled Trials in Pediatric Critical Care: The Perspectives of Trialists

OBJECTIVES

Clinical research is a complex scientific and social enterprise. Our objective was to identify strategies that pediatric critical care trialists consider acceptable, feasible, and effective to improve the design and conduct randomized controlled trials in pediatric critical care.

DESIGN

Qualitative descriptive study using semistructured individual interviews.

SUBJECTS

We interviewed 26 pediatric critical care researchers from seven countries who have published a randomized controlled trial (2005-2015). We used purposive sampling to achieve diversity regarding researcher characteristics and randomized controlled trial characteristics.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Most participants (24 [92%]) were from high-income countries, eight (31%) had published more than one randomized controlled trial, 17 (65%) had published a multicenter randomized controlled trial, and eight (31%) had published a multinational randomized controlled trial. An important theme was "building communities"-groups of individuals with similar interests, shared experiences, and common values, bound by professional and personal relationships. Participants described a sense of community as a source of motivation and encouragement and as a means to larger, more rigorous trials, increasing researcher and clinician engagement and maintaining enthusiasm. Strategies to build communities stressed in-person interactions (both professional and social), capable leadership, and trust. Another important theme was "getting started." Participants highlighted the importance of formal research training and high-quality experiential learning through collaboration on other's projects, guided by effective mentorship. Also important was "working within the system"-ensuring academic credit for a range of contributions, not only for the principal investigator role. The longitudinal notion of "building on success" was also underscored as a cross-cutting theme.

CONCLUSIONS

Coordinated, deliberate actions to build community and ensure key training and practical experiences for new investigators may strengthen the research enterprise in pediatric critical care. These strategies, potentially in combination with other novel approaches, may vitalize clinical research in this field.

What really matters for successful research environments? A realist synthesis

CONTEXT

Research environments, or cultures, are thought to be the most influential predictors of research productivity. Although several narrative and systematic reviews have begun to identify the characteristics of research-favourable environments, these reviews have ignored the contextual complexities and multiplicity of environmental characteristics.

OBJECTIVES

The current synthesis adopts a realist approach to explore what interventions work for whom and under what circumstances.

METHODS

We conducted a realist synthesis of the international literature in medical education, education and medicine from 1992 to 2016, following five stages: (i) clarifying the scope; (ii) searching for evidence; (iii) assessing quality; (iv) extracting data, and (v) synthesising data.

RESULTS

We identified numerous interventions relating to research strategy, people, income, infrastructure and facilities (IIF), and collaboration. These interventions resulted in positive or negative outcomes depending on the context and mechanisms fired. We identified diverse contexts at the individual and institutional levels, but found that disciplinary contexts were less influential. There were a multiplicity of positive and negative mechanisms, along with three cross-cutting mechanisms that regularly intersected: time; identity, and relationships. Outcomes varied widely and included both positive and negative outcomes across subjective (e.g. researcher identity) and objective (e.g. research quantity and quality) domains.

CONCLUSIONS

The interplay among mechanisms and contexts is central to understanding the outcomes of specific interventions, bringing novel insights to the literature. Researchers, research leaders and research organisations should prioritise the protection of time for research, enculturate researcher identities, and develop collaborative relationships to better foster successful research environments. Future research should further explore the interplay among time, identity and relationships.

Contributions of a survey and retrospective cohort study to the planning of a randomised controlled trial of corticosteroids in the treatment of paediatric septic shock

Background

Randomised controlled trials (RCTs) are challenging to conduct in a paediatric critical-care environment. Background work, including surveys and observational studies, is often used to determine disease estimates, sample sizes and design protocols when planning such RCTs. Our objective was to determine the necessity of performing a survey or a retrospective chart review or both when planning an RCT on corticosteroids in the treatment of paediatric septic shock.

Methods

We compared information on corticosteroid use for moderate to severe paediatric septic shock obtained from a survey of physician beliefs and stated practices with that obtained from a retrospective cohort study. The survey was conducted between February and March 2012 and the retrospective study included children from birth to 17 years of age admitted from January 2010 to June 2011. The survey and the retrospective study were conducted at four academic tertiary care centres in Canada.

Results

Survey responses from 23 physicians and retrospective data from 81 septic shock patients were included. The survey identified time to discontinuation of vasoactive infusions as the most feasible and clinically important outcome for an RCT on corticosteroids for paediatric septic shock. The retrospective chart review provided means and standard deviations for the suggested primary outcome, from which we could estimate sample sizes and justify the minimal clinically important difference. The survey found that physicians believe that patients with severe septic shock were most likely to benefit from corticosteroid administration but the majority stated they would be unwilling to randomise such patients, suggesting a lack of individual physician equipoise. The combined information from the survey and retrospective study suggested that enrolment of patients with moderate septic shock would be more feasible but that strategies would still have to be implemented to prevent open-label corticosteroid use.

Conclusions

The survey provided valuable information on the choice of primary outcome, target population and physician equipoise. The retrospective study provided estimates of patient numbers, the minimal clinically important difference, evidence for community equipoise and physician practice patterns. Strong consideration should be given to performing both types of studies prior to conducting RCTs in paediatric critical-care environments.

Barriers to participation in surgical randomized controlled trials in pediatric urology: A qualitative study of key stakeholder perspectives

INTRODUCTION

Randomized controlled trials (RCTs) are considered the gold standard for assessing treatment efficacy. However, pediatric surgical RCTs have been limited in their ability to recruit patients. The purpose of this study was to identify barriers and motivators to pediatric participation in surgical RCTs.

METHODS

We conducted a series of two focus groups with parents and one focus group with urology providers for children aged <2 years of age with a diagnosis of Society for Fetal Urology grade 3 or 4 hydronephrosis. We then administered a survey to referring pediatricians based on the initial analysis of focus group findings. Theme analysis was used for all qualitative transcribed text data obtained from focus groups and open-ended survey questions using team-based inductive approaches. Descriptive statistics were obtained for the remainder of the provider survey.

RESULTS

Using qualitative text from stakeholders (n = 38) we identified four key themes across the data: responsibility to my child; responsibility to my patient; responsibility to the field; and irreversibility of surgery. Participants felt there was an obligation to be informed of relevant scientific research within a clinic research culture. However, there remains a disconnect for parents between randomized research studies that may ultimately benefit their child, depending on their age and concern their child is being treated as a 'guinea pig'. Some parents were willing to participate in RCTs but all were more open to participate in an observational study where the treatment decisions were felt to be under their control even when there was no "right answer" or multiple equivalent options for treatment. There was mixed opinion across the parents and providers whether research trial education and enrollment should be provided by the pediatrician or urologist. Active physician decisions were seen as critical within the context of a long term clinical relationship and provision of information of risks and benefits without pressure were considered essential for ethical research by both parents and providers.

CONCLUSION

While some parents are open to participation in surgical RCTs, providers and parents of children with hydronephrosis feel discomfort with the element of chance in surgical randomized trials. Parents and providers are more likely to participate in observational studies where treatment decisions may be made jointly by the physician and the parent. These findings suggest that pragmatic trial strategies with the option for participation in an observational cohort may improve recruitment of pediatric patients into surgical clinical trials.

The natural history of conducting and reporting clinical trials: interviews with trialists

BACKGROUND

To investigate the nature of the research process as a whole, factors that might influence the way in which research is carried out, and how researchers ultimately report their findings.

METHODS

Semi-structured qualitative telephone interviews with authors of trials, identified from two sources: trials published since 2002 included in Cochrane systematic reviews selected for the ORBIT project; and trial reports randomly sampled from 14,758 indexed on PubMed over the 12-month period from August 2007 to July 2008.

RESULTS

A total of 268 trials were identified for inclusion, 183 published since 2002 and included in the Cochrane systematic reviews selected for the ORBIT project and 85 randomly selected published trials indexed on PubMed. The response rate from researchers in the former group was 21% (38/183) and in the latter group was 25% (21/85). Overall, 59 trialists were interviewed from the two different sources. A number of major but related themes emerged regarding the conduct and reporting of trials: establishment of the research question; identification of outcome variables; use of and adherence to the study protocol; conduct of the research; reporting and publishing of findings. Our results reveal that, although a substantial proportion of trialists identify outcome variables based on their clinical experience and knowing experts in the field, there can be insufficient reference to previous research in the planning of a new trial. We have revealed problems with trial recruitment: not reaching the target sample size, over-estimation of recruitment potential and recruiting clinicians not being in equipoise. We found a wide variation in the completeness of protocols, in terms of detailing study rationale, outlining the proposed methods, trial organisation and ethical considerations.

CONCLUSION

Our results confirm that the conduct and reporting of some trials can be inadequate. Interviews with researchers identified aspects of clinical research that can be especially challenging: establishing appropriate and relevant outcome variables to measure, use of and adherence to the study protocol, recruiting of study participants and reporting and publishing the study findings. Our trialists considered the prestige and impact factors of academic journals to be the most important criteria for selecting those to which they would submit manuscripts.

Randomized controlled trials in pediatric critical care: a scoping review

Introduction

Evidence from randomized controlled trials (RCTs) is required to guide treatment of critically ill children, but the number of RCTs available is limited and the publications are often difficult to find. The objectives of this review were to systematically identify RCTs in pediatric critical care and describe their methods and reporting.

Methods

We searched MEDLINE, EMBASE, LILACS and CENTRAL (from inception to April 16, 2013) and reference lists of included RCTs and relevant systematic reviews. We included published RCTs administering any intervention to children in a pediatric ICU. We excluded trials conducted in neonatal ICUs, those enrolling exclusively preterm infants, and individual patient crossover trials. Pairs of reviewers independently screened studies for eligibility, assessed risk of bias, and abstracted data. Discrepancies were resolved by consensus.

Results

We included 248 RCTs: 45 (18%) were multicentered and 14 (6%) were multinational. Trials most frequently enrolled both medical and surgical patients (43%) but postoperative cardiac surgery was the single largest population studied (19%). The most frequently evaluated types of intervention were medications (63%), devices (11%) and nutrition (8%). Laboratory or physiological measurements were the most frequent type of primary outcomes (18%). Half of these trials (50%) reported blinding. Of the 107 (43%) trials that reported an a priori sample size, 34 (32%) were stopped early. The median number of children randomized per trial was 49 and ranged from 6 to 4,947. The frequency of RCT publications increased at a mean rate of 0.7 RCTs per year (P<0.001) from 1 to 20 trials per year.

Conclusions

This scoping review identified the available RCTs in pediatric critical care and made them accessible to clinicians and researchers (http://epicc.mcmaster.ca). Most focused on medications and intermediate or surrogate outcomes, were single-centered and were conducted in North America and Western Europe. The results of this review underscore the need for trials with rigorous methodology, appropriate outcome measures, and improved quality of reporting to ensure that high quality evidence exists to support clinical decision-making in this vulnerable population.

Education in health research methodology: use of a wiki for knowledge translation

Introduction

A research-practice gap exists between what is known about conducting methodologically rigorous randomized controlled trials (RCTs) and what is done. Evidence consistently shows that pediatric RCTs are susceptible to high risk of bias; therefore novel methods of influencing the design and conduct of trials are required. The objective of this study was to develop and pilot test a wiki designed to educate pediatric trialists and trainees in the principles involved in minimizing risk of bias in RCTs. The focus was on preliminary usability testing of the wiki.

Methods

The wiki was developed through adaptation of existing knowledge translation strategies and through tailoring the site to the identified needs of the end-users. The wiki was evaluated for usability and user preferences regarding the content and formatting. Semi-structured interviews were conducted with 15 trialists and systematic reviewers, representing varying levels of experience with risk of bias or the conduct of trials. Data were analyzed using content analysis.

Results

Participants found the wiki to be well organized, easy to use, and straightforward to navigate. Suggestions for improvement tended to focus on clarification of the text or on esthetics, rather than on the content or format. Participants liked the additional features of the site that were supplementary to the text, such as the interactive examples, and the components that focused on practical applications, adding relevance to the theory presented. While the site could be used by both trialists and systematic reviewers, the lack of a clearly defined target audience caused some confusion among participants.

Conclusions

Participants were supportive of using a wiki as a novel educational tool. The results of this pilot test will be used to refine the risk of bias wiki, which holds promise as a knowledge translation intervention for education in medical research methodology.