The evidence base for interventions delivered to children in primary care: an overview of cochrane systematic reviews

Fake news and fake research: Why meta-research matters more than ever

Research is in a crisis of credibility, and this is to the peril of all paediatricians. Billions of dollars are being wasted each year because research is not planned, badly conducted or poorly reported, and this is on a background of rapidly reducing research budgets. How can paediatricians, families and patients make informed treatment choices if the evidence base is absent or not trustworthy? This article discusses why meta-research now matters more than ever, how it can help solve this crisis of credibility and how this should lead to more efficient and effective clinical care. The field of meta-research or research-on-research is the ultimate big picture approach to identifying and solving issues of bias, error, misconduct and waste in research. Meta-researchers value authenticity over aesthetics and quality over quantity. The utility of meta-research does not rely on accusations or critical assessments of individual research, but through highlighting where and how the scientific method and research standards across all fields can be improved. Meta-researchers study, analyse and critique the research pathway, focusing on elements such as methods (how to conduct), evaluation (how to test), reporting (how to communicate), reproducibility (how to verify) and incentives (how to reward). In the current climate it is now more critical than ever that we make use of meta-research and prioritise high-quality high-impact research, ultimately leading to improved patient outcomes.

Patient and public involvement in the design of clinical trials: An overview of systematic reviews

BACKGROUND

Funders encourage lay-volunteer inclusion in research. There are controversy and resistance, given concerns of role confusion, exploratory methods, and limited evidence about what value lay-volunteers bring to research. This overview explores these areas.

METHODS

Eleven databases were searched without date or language restrictions for systematic reviews of public and patient involvement (PPI) in clinical trials design. This systematic overview of PPI included 27 reviews from which areas of good and bad practice were identified. Strengths, weaknesses, opportunities, and threats of PPI were explored through use of meta-narrative analysis.

RESULTS

Inclusion criteria were met by 27 reviews ranging in quality from high (n = 7), medium (n = 14) to low (n = 6) reviews. Reviews were assessed using CERQUAL NICE, CASP for qualitative research and CASP for systematic reviews. Four reviews report risk of bias. Public involvement roles were primarily in agenda setting, steering committees, ethical review, protocol development, and piloting. Research summaries, follow-up, and dissemination contained PPI, with lesser involvement in data collection, analysis, or manuscript authoring. Trialists report difficulty in finding, retaining, and reimbursing volunteers. Respectful inclusion, role recognition, mutual flexibility, advance planning, and sound methods were reported as facilitating public involvement in research. Public involvement was reported to have increased the quantity and quality of patient relevant priorities and outcomes, enrollment, funding, design, implementation, and dissemination. Challenges identified include lack of clarity within common language, roles, and research boundaries, while logistical needs include extra time, training, and funding. Researchers report struggling to report involvement and avoid tokenism.

CONCLUSIONS

Involving patients and the public in clinical trials design can be beneficial but requires resources, preparation, training, flexibility, and time. Issues to address include reporting deficits for risk of bias, study quality, and conflicts of interests. We need to address these tensions and improve dissemination strategies to increase PPI and health literacy.

Reaching beyond the review of research evidence: a qualitative study of decision making during the development of clinical practice guidelines for disease prevention in healthcare

Background

The judgment and decision making process during guideline development is central for producing high-quality clinical practice guidelines, but the topic is relatively underexplored in the guideline research literature. We have studied the development process of national guidelines with a disease-prevention scope produced by the National board of Health and Welfare (NBHW) in Sweden. The NBHW formal guideline development model states that guideline recommendations should be based on five decision-criteria: research evidence; curative/preventive effect size, severity of the condition; cost-effectiveness; and ethical considerations. A group of health profession representatives (i.e. a prioritization group) was assigned the task of ranking condition-intervention pairs for guideline recommendations, taking into consideration the multiple decision criteria. The aim of this study was to investigate the decision making process during the two-year development of national guidelines for methods of preventing disease.

Methods

A qualitative inductive longitudinal case study approach was used to investigate the decision making process. Questionnaires, non-participant observations of nine two-day group meetings, and documents provided data for the analysis. Conventional and summative qualitative content analysis was used to analyse data.

Results

The guideline development model was modified ad-hoc as the group encountered three main types of dilemmas: high quality evidence vs. low adoptability of recommendation; insufficient evidence vs. high urgency to act; and incoherence in assessment and prioritization within and between four different lifestyle areas. The formal guideline development model guided the decision-criteria used, but three new or revised criteria were added by the group: ‘clinical knowledge and experience’, ‘potential guideline consequences’ and ‘needs of vulnerable groups’. The frequency of the use of various criteria in discussions varied over time. Gender, professional status, and interpersonal skills were perceived to affect individuals’ relative influence on group discussions.

Conclusions

The study shows that guideline development groups make compromises between rigour and pragmatism. The formal guideline development model incorporated multiple aspects, but offered few details on how the different criteria should be handled. The guideline development model devoted little attention to the role of the decision-model and group-related factors. Guideline development models could benefit from clarifying the role of the group-related factors and non-research evidence, such as clinical experience and ethical considerations, in decision-processes during guideline development.

Mind the gap in clinical trials: A participatory action analysis with citizen collaborators

What are the strengths, gaps, expectations, and barriers to research engagement in clinical trials as communicated through social media? Clinical trials test treatments to provide reliable information for safety and effectiveness. Trials are building blocks in which what is learned in earlier research can be used to improve treatments, compare alternatives, and improve quality of life. For 20 years, the percentages of clinical trials volunteers have decreased whereas the costs of running clinical trials have multiplied. Participants enroll in trials to access latest treatments, to help others, and to advance science, but there is growing unrest. The priorities of those running the trials differ from those of the participants, and the roles for public research involvement lack clarity. Changes to bridge these gaps in the research culture are proposed through the use of participatory action research (PAR) in which stakeholders collaborate to improve research methodology, galvanize citizen participation, multiply health knowledge, problem-solve barriers to access, and explore the value of research volunteers as collaborators. PAR enabled the inclusion of citizens as full collaborators. Social media data were gathered for 120 days until saturation was reached. De-identified data were organized into a Strengths Weaknesses, Opportunities and Threats framework and coded into themes for analysis. After the analysis, the authors prioritized potential solutions for improving research engagement. Strengths and opportunities remained constant through trial phases, disease burdens, and interventions. Threats included alienation, litigation, disparity, and shaming. Poor management and barriers to inclusion were identified as weaknesses. Opportunities included improving resource management and information quality. Barriers were minimized when relationships between staff and participants were inclusive, respectful, tolerant, and open to change. Participants' communications ranged from fulfillment through trial involvement to disparities and rights violations. PAR provides a safe space without power imbalances in which researchers and citizen worked as equals rather than as researchers and objects of research.

Patient Safety Incidents Involving Sick Children in Primary Care in England and Wales: A Mixed Methods Analysis

BACKGROUND

The UK performs poorly relative to other economically developed countries on numerous indicators of care quality for children. The contribution of iatrogenic harm to these outcomes is unclear. As primary care is the first point of healthcare contact for most children, we sought to investigate the safety of care provided to children in this setting.

METHODS AND FINDINGS

We undertook a mixed methods investigation of reports of primary care patient safety incidents involving sick children from England and Wales' National Reporting and Learning System between 1 January 2005 and 1 December 2013. Two reviewers independently selected relevant incident reports meeting prespecified criteria, and then descriptively analyzed these reports to identify the most frequent and harmful incident types. This was followed by an in-depth thematic analysis of a purposive sample of reports to understand the reasons underpinning incidents. Key candidate areas for strengthening primary care provision and reducing the risks of systems failures were then identified through multidisciplinary discussions. Of 2,191 safety incidents identified from 2,178 reports, 30% (n = 658) were harmful, including 12 deaths and 41 cases of severe harm. The children involved in these incidents had respiratory conditions (n = 387; 18%), injuries (n = 289; 13%), nonspecific signs and symptoms, e.g., fever (n = 281; 13%), and gastrointestinal or genitourinary conditions (n = 268; 12%), among others. Priority areas for improvement included safer systems for medication provision in community pharmacies; triage processes to enable effective and timely assessment, diagnosis, and referral of acutely sick children attending out-of-hours services; and enhanced communication for robust safety netting between professionals and parents. The main limitations of this study result from underreporting of safety incidents and variable data quality. Our findings therefore require further exploration in longitudinal studies utilizing case review methods.

CONCLUSIONS

This study highlights opportunities to reduce iatrogenic harm and avoidable child deaths. Globally, healthcare systems with primary-care-led models of delivery must now examine their existing practices to determine the prevalence and burden of these priority safety issues, and utilize improvement methods to achieve sustainable improvements in care quality.

Randomized controlled trials in pediatric critical care: a scoping review

Introduction

Evidence from randomized controlled trials (RCTs) is required to guide treatment of critically ill children, but the number of RCTs available is limited and the publications are often difficult to find. The objectives of this review were to systematically identify RCTs in pediatric critical care and describe their methods and reporting.

Methods

We searched MEDLINE, EMBASE, LILACS and CENTRAL (from inception to April 16, 2013) and reference lists of included RCTs and relevant systematic reviews. We included published RCTs administering any intervention to children in a pediatric ICU. We excluded trials conducted in neonatal ICUs, those enrolling exclusively preterm infants, and individual patient crossover trials. Pairs of reviewers independently screened studies for eligibility, assessed risk of bias, and abstracted data. Discrepancies were resolved by consensus.

Results

We included 248 RCTs: 45 (18%) were multicentered and 14 (6%) were multinational. Trials most frequently enrolled both medical and surgical patients (43%) but postoperative cardiac surgery was the single largest population studied (19%). The most frequently evaluated types of intervention were medications (63%), devices (11%) and nutrition (8%). Laboratory or physiological measurements were the most frequent type of primary outcomes (18%). Half of these trials (50%) reported blinding. Of the 107 (43%) trials that reported an a priori sample size, 34 (32%) were stopped early. The median number of children randomized per trial was 49 and ranged from 6 to 4,947. The frequency of RCT publications increased at a mean rate of 0.7 RCTs per year (P<0.001) from 1 to 20 trials per year.

Conclusions

This scoping review identified the available RCTs in pediatric critical care and made them accessible to clinicians and researchers (http://epicc.mcmaster.ca). Most focused on medications and intermediate or surrogate outcomes, were single-centered and were conducted in North America and Western Europe. The results of this review underscore the need for trials with rigorous methodology, appropriate outcome measures, and improved quality of reporting to ensure that high quality evidence exists to support clinical decision-making in this vulnerable population.

Coverage of mental health and substance misuse topics in the Cochrane review system

Aims. To assess the breadth of mental and substance coverage in the Cochrane review system. Methods. All mental health and substance entries were identified from the 2005 to April 2012 Cochrane Database of Systematic Reviews. Results. A total of 1019 entries focused on mental health or substance misuse, with 698 (68.5%) being completed reviews. One out of every five entries focused on serious mental illness/psychosis. Systematic reviews addressing unipolar depression, dementia and certain substance disorders also appeared well-represented. In contrast, a number of impairing disorders frequently seen in practice received less attention, with bipolar disorder, obsessive compulsive disorder (OCD), post-traumatic stress disorder (PTSD) and autism spectrum disorders each accounting for less than 2% of the entries. The majority of interventions reviewed involved medication (57.1%), although this was not the case for a number of childhood-onset disorders. Some diagnostic areas (sleep, anxiety, mood and substance) were addressed by multiple Cochrane review groups (CRGs). Conclusions. The Cochrane Collaboration is well poised to be a strong guiding influence to those seeking to employ evidence-based mental health care. Broadening its diagnostic coverage and diversifying types of intervention reviewed would probably further maximize its impact. A more centralized and directed approach of prioritizing topics could help ensure more comprehensive coverage.

Health services for children in western Europe

Western European health systems are not keeping pace with changes in child health needs. Non-communicable diseases are increasingly common causes of childhood illness and death. Countries are responding to changing needs by adapting child health services in different ways and useful insights can be gained through comparison, especially because some have better outcomes, or have made more progress, than others. Although overall child health has improved throughout Europe, wide inequities remain. Health services and social and cultural determinants contribute to differences in health outcomes. Improvement of child health and reduction of suffering are achievable goals. Development of systems more responsive to evolving child health needs is likely to necessitate reconfiguring of health services as part of a whole-systems approach to improvement of health. Chronic care services and first-contact care systems are important aspects. The Swedish and Dutch experiences of development of integrated systems emphasise the importance of supportive policies backed by adequate funding. France, the UK, Italy, and Germany offer further insights into chronic care services in different health systems. First-contact care models and the outcomes they deliver are highly variable. Comparisons between systems are challenging. Important issues emerging include the organisation of first-contact models, professional training, arrangements for provision of out-of-hours services, and task-sharing between doctors and nurses. Flexible first-contact models in which child health professionals work closely together could offer a way to balance the need to provide expertise with ready access. Strategies to improve child health and health services in Europe necessitate a whole-systems approach in three interdependent systems-practice (chronic care models, first-contact care, competency standards for child health professionals), plans (child health indicator sets, reliable systems for capture and analysis of data, scale-up of child health research, anticipation of future child health needs), and policy (translation of high-level goals into actionable policies, open and transparent accountability structures, political commitment to delivery of improvements in child health and equity throughout Europe).

Quality of reporting and evidence in American Academy of Pediatrics guidelines

OBJECTIVES

The primary objectives were to evaluate the quality of development and reporting of American Academy of Pediatrics (AAP) guidelines and to determine the level of evidence underlying the recommendations.

METHODS

Two reviewers scored each guideline by using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) instrument and determined the level of evidence for each recommendation in each guideline. Subgroup analyses compared AAP guidelines published before and after key changes in AAP guideline development policy and compared internal with endorsed guidelines.

RESULTS

For the 28 current guidelines, the highest average scores on AGREE-II were in scope and purpose (75%) and clarity of presentation (73%). The lowest average scores were in editorial independence (17%) and applicability (30%). The only domain that improved after AAP policy updates was editorial independence (P = .01). Of the 190 treatment recommendations, 43% were based on experimental studies, 30% on observational studies, and 27% on expert opinion or no reference. Compared with early guidelines, late guidelines included a higher proportion of treatment recommendations based on experimental studies (P = .05).

CONCLUSIONS

There was no clear improvement in the quality of development and reporting of AAP clinical practice guidelines over time. Routine application of AGREE-II to guideline development could enhance guideline quality. The proportion of guideline recommendations based on experimental evidence has increased slightly over time. Pediatric research agendas should be matched to vital gaps in the evidence underlying pediatric guidelines.

Prioritizing areas for quality marker development in children in UK general practice: extending the use of the nominal group technique

BACKGROUND

There is a deficiency in the ability to measure the quality of care of children in primary care and there is no professional consensus in UK general practice regarding which quality markers should be used.

OBJECTIVES

To prioritize clinical areas on which to focus quality marker development in paediatric primary care and to describe the challenges in generating professional consensus.

METHODS

We convened an expert panel of GPs with a special interest in child health and using the nominal group technique (NGT), a well-established structured, multistep facilitated group meeting technique, we generated consensus around the key clinical areas to focus quality marker development.

RESULTS

Twelve GPs participated in the expert panel. The eight items agreed by panellists as most important were early recognition of serious illness, whole practice involvement in safeguarding, health promotion, mental health, evidence-based management of common conditions, child and carer friendliness and safe and cost-effective prescribing. Panel members struggled to balance the broad clinical areas while attempting to focus on specific areas that are important. The main challenges included managing panel uncertainty, effective organization, presentation of items for review and group inclination to 'include everything'.

CONCLUSIONS

This is the first consensus study of UK GPs to identify key areas to target quality marker development in children. By using the NGT, we have highlighted front-line health care professionals' priorities to improve the quality of care of children and identified the benefits and challenges of developing consensus in a broad topic area.