Prioritizing areas for quality marker development in children in UK general practice: extending the use of the nominal group technique

Patient, Caregiver, and Clinician Participation in Prioritization of Research Questions in Pediatric Hospital Medicine

IMPORTANCE

The research agenda in pediatric hospital medicine has seldom considered the perspectives of young people, parents and caregivers, and health care professionals. Their perspectives may be useful in identifying questions on topics for research.

OBJECTIVE

To prioritize unanswered research questions in pediatric hospital medicine from the perspectives of young people, parents/caregivers, and health care professionals.

DESIGN, SETTING, AND PARTICIPANTS

Between August 4, 2020, and August 19, 2021, two online surveys and a virtual workshop were conducted, using modified Delphi technique and nominal group technique. Young people, parents/caregivers, and health care professionals with experiences in pediatric hospital medicine in Canada were included.

INTERVENTIONS

The established James Lind Alliance Priority Setting Partnership method was used. In phase 1, a survey collected unanswered questions regarding pediatric hospital medicine via 3 open-ended questions. Survey responses were used to develop summary questions that went through an evidence-checking process. Unanswered questions were brought to a phase 2 interim prioritization survey. The top 10 unanswered research questions in pediatric hospital medicine were established at the final priority setting workshop.

MAIN OUTCOMES AND MEASURES

Survey responses, top 10 research questions.

RESULTS

The phase 1 survey was completed by 188 participants (148 of 167 [89%] females; 17 of 167 [10%] males; mean [SD] age, 39.5 [12.4] years) and generated 495 unanswered research questions and comments, of which 58 were deemed out of scope. The remaining 437 responses were grouped into themes (eg, communication, shared decision-making, health service delivery, and health service management) and then refined to 75 unanswered research questions. Of these 75, only 4 questions had sufficient evidence. To make the number of questions in phase 2 manageable, 21 questions submitted by only 1 respondent were eliminated. Fifty unanswered research questions were included in the phase 2 survey, which was completed by 201 participants (165 of 186 [89%] females; 19 of 186 [10%] males; mean [SD] age, 40.0 [11.0] years). A short list of 16 questions-the top 10 questions from patient partners (youths, parents/caregivers) and clinicians-was presented at the final priority setting workshop and the top 10 questions were prioritized. The top 10 questions focused on the care of special inpatient populations (eg, children with medical complexity), communication, shared decision-making, support strategies in the hospital, mental health supports, shortening length of stay, and supporting Indigenous patients, parents/caregivers, and families.

CONCLUSIONS AND RELEVANCE

This patient-oriented pediatric hospital medicine priority setting partnership identified the most important unanswered research questions focused on the care of children in the hospital. These questions provide a possible roadmap for research on areas deemed important to young people, parents/caregivers, and clinicians.

Residents as Research Subjects: Balancing Resident Education and Contribution to Advancing Educational Innovations

BACKGROUND

Research in education advances knowledge and improves learning, but the literature does not define how to protect residents' rights as subjects in studies or how to limit the impact of their participation on their clinical training.

OBJECTIVE

We aimed to develop a consensual framework on how to include residents as participants in education research, with the dual goal of protecting their rights and promoting their contributions to research.

METHODS

A nominal group technique approach was used to structure 3 iterative meetings held with the pre-existing residency training program committee and 7 invited experts between September 2018 and April 2019. Thematic text analysis was conducted to prepare a final report, including recommendations.

RESULTS

Five themes, each with recommendations, were identified: (1) Freedom of participation: participation, non-participation, or withdrawal from a study should not interfere with teacher-learner relationship (recommendation: improve recruitment and consent forms); (2) Avoidance of over-solicitation (recommendation: limit the number of ongoing studies); (3) Management of time dedicated to participation in research (recommendations: schedule and proportion of time for study participation); (4) Emotional safety (recommendation: requirement for debriefing and confidential counseling); and (5) Educational safety: data collected during a study should not influence clinical assessment of the resident (recommendation: principal investigator should not be involved in the evaluation process of learners in clinical rotation).

CONCLUSIONS

Our nominal group technique approach resulted in raising 5 specific issues about freedom of participation of residents in research in medical education, over-solicitation, time dedicated to research, emotional safety, and educational safety.

Development of a Symptom-Focused Model to Guide the Prescribing of Antipsychotics in Children and Adolescents: Results of the First Phase of the Safer Use of Antipsychotics in Youth (SUAY) Clinical Trial

OBJECTIVE

To develop a new approach to prescribing guidelines as part of a pragmatic trial, Safer Use of Antipsychotics in Youth (SUAY; ClinicalTrials.gov Identifier: NCT03448575), which supports prescribers in delivering high-quality mental health care to youths.

METHOD

A nominal group technique was used to identify first- to nth-line treatments for target symptoms and potential diagnoses. The panel included US pediatricians, child and adolescent psychiatrists, and psychopharmacology experts. Meeting materials included information about Medicaid review programs, systematic reviews, prescribing guidelines, and a description of the pragmatic trial. Afterward, a series of 4 webinar discussions were held to achieve consensus on recommendations.

RESULTS

The panel unanimously agreed that the guideline should focus on target symptoms rather than diagnoses. Guidance included recommendations for first- to nth-line treatment of target mental health symptoms, environmental factors to be addressed, possible underlying diagnoses that should first be considered and ruled out, and general considerations for pharmacological and therapeutic treatments.

CONCLUSION

Prescribing guidelines are often ignored because they do not incorporate the real-world availability of first-line psychosocial treatments, comorbid conditions, and clinical complexity. Our approach addresses some of these concerns. If the approach proves successful in our ongoing pragmatic trial, Safer Use of Antipsychotics in Youth (SUAY), it may serve as a model to state Medicaid programs and health systems to support clinicians in delivering high-quality mental health care to youths.

CLINICAL TRIAL REGISTRATION INFORMATION

Safer Use of Antipsychotics in Youth; http://clinicaltrials.gov/; NCT03448575.

Setting a prioritized agenda to drive speech-language therapy research in health

BACKGROUND

Prioritized research agendas are viewed internationally as an important method for ensuring that health research meets actual areas of clinical need. There is growing evidence for speech-language therapy-prioritized research agendas, particularly in disorder-specific areas. However, there are few general research priority agendas to guide speech-language therapy research.

AIMS

To collaboratively develop a prioritized research agenda for an Australian public health context with clinical speech-language therapists (SLTs), academic SLTs and consumers of speech-language therapy services.

METHODS & PROCEDURES

An initial stimulus list of potential research areas for prioritization was collected from SLTs via an online survey. Two categories (service delivery and expanded scope of practice) were selected from this list for prioritization due to their relevance across multiple health services. The Nominal Group Technique (NGT) was used to develop a prioritized research agenda for each of the two categories. One NGT session was conducted with each of the three participant groups (clinical SLTs, academic SLTs, consumers) for each category (total NGT sessions = six). The prioritization data for each group within each category were summed to give a single, ranked prioritized research agenda for each category.

OUTCOMES & RESULTS

Two prioritized research agendas were developed. Within each agenda, SLTs and consumers prioritized a need for more research in areas related to specific practice areas (e.g., Alternative and Augmentative Communication, Communication Partner Training), as well as broader professional issues (e.g., telehealth, working with culturally and linguistically diverse families).

CONCLUSIONS & IMPLICATIONS

The current findings support the need for funding proposals and targeted projects that address these identified areas of need.

WHAT THIS PAPER ADDS

What is already known on this subject Evidence-based practice is a critical component of SLT practice. There is often a disconnect between the research evidence generated and areas of clinical need, and in some areas a lack of evidence. Prioritized research agendas can help drive research in areas of clinical need. What this paper adds to existing knowledge A collaborative, prioritized SLT research agenda was developed using the NGT according to the views of clinical SLTs, academic SLTs and consumers of speech-language therapy services in a conglomerate of public health services. SLTs and consumers identified a need for further research in specific areas of SLT practice as well as broader emerging professional issues What are the potential or actual clinical implications of this work? Targeted research projects funded on a large scale are required to address these identified areas of need. Other health services around the world could replicate this prioritization process to drive research in areas of clinical need.

Consumer-Guided Development of an Engagement-Facilitation Intervention for Increasing Uptake and Adherence for Self-Guided Web-Based Mental Health Programs: Focus Groups and Online Evaluation Survey

Background

Self-guided web-based mental health programs are effective in treating and preventing mental health problems. However, current engagement with these programs in the community is suboptimal, and there is limited evidence indicating how to increase the use of existing evidence-based programs.

Objective

This study aims to investigate the views of people with lived experience of depression and anxiety on factors influencing their engagement with self-guided web-based mental health (e–mental health) programs and to use these perspectives to develop an engagement-facilitation intervention (EFI) to increase engagement (defined as both uptake and adherence) with these programs.

Methods

A total of 24 community members (female=21; male=3) with lived experience of depression and anxiety or depression or anxiety alone participated in 1 of 4 focus groups discussing the factors influencing their engagement with self-guided e–mental health programs and the appearance, delivery mode, and functionality of content for the proposed EFI. A subsequent evaluation survey of the focus group participants (n=14) was conducted to evaluate the resultant draft EFI. Data were thematically analyzed using both inductive and deductive qualitative methods.

Results

Participants suggested that the critical component of an EFI was information that would challenge personal barriers to engagement, including receiving personalized symptom feedback, information regarding the program’s content or effectiveness and data security, and normalization of using e–mental health programs (eg, testimonials). Reminders, rewards, feedback about progress, and coaching were all mentioned as facilitating adherence.

Conclusions

EFIs have the potential to improve community uptake of e–mental health programs. They should focus on providing information on the content and effectiveness of e–mental health programs and normalizing their use. Given that the sample comprised predominantly young females, this study may not be generalizable to other population groups. There is a strong value in involving people with a lived experience in the design and development of EFIs to maximize their effectiveness.

Early morbidities following paediatric cardiac surgery: a mixed-methods study

Background

Over 5000 paediatric cardiac surgeries are performed in the UK each year and early survival has improved to > 98%.

Objectives

We aimed to identify the surgical morbidities that present the greatest burden for patients and health services and to develop and pilot routine monitoring and feedback.

Design and setting

Our multidisciplinary mixed-methods study took place over 52 months across five UK paediatric cardiac surgery centres.

Participants

The participants were children aged < 17 years.

Methods

We reviewed existing literature, ran three focus groups and undertook a family online discussion forum moderated by the Children’s Heart Federation. A multidisciplinary group, with patient and carer involvement, then ranked and selected nine key morbidities informed by clinical views on definitions and feasibility of routine monitoring. We validated a new, nurse-administered early warning tool for assessing preoperative and postoperative child development, called the brief developmental assessment, by testing this among 1200 children. We measured morbidity incidence in 3090 consecutive surgical admissions over 21 months and explored risk factors for morbidity. We measured the impact of morbidities on quality of life, clinical burden and costs to the NHS and families over 6 months in 666 children, 340 (51%) of whom had at least one morbidity. We developed and piloted methods suitable for routine monitoring of morbidity by centres and co-developed new patient information about morbidities with parents and user groups.

Results

Families and clinicians prioritised overlapping but also different morbidities, leading to a final list of acute neurological event, unplanned reoperation, feeding problems, renal replacement therapy, major adverse events, extracorporeal life support, necrotising enterocolitis, surgical infection and prolonged pleural effusion. The brief developmental assessment was valid in children aged between 4 months and 5 years, but not in the youngest babies or 5- to 17-year-olds. A total of 2415 (78.2%) procedures had no measured morbidity. There was a higher risk of morbidity in neonates, complex congenital heart disease, increased preoperative severity of illness and with prolonged bypass. Patients with any morbidity had a 6-month survival of 81.5% compared with 99.1% with no morbidity. Patients with any morbidity scored 5.2 points lower on their total quality of life score at 6 weeks, but this difference had narrowed by 6 months. Morbidity led to fewer days at home by 6 months and higher costs. Extracorporeal life support patients had the lowest days at home (median: 43 days out of 183 days) and highest costs (£71,051 higher than no morbidity).

Limitations

Monitoring of morbidity is more complex than mortality, and hence this requires resources and clinician buy-in.

Conclusions

Evaluation of postoperative morbidity provides important information over and above 30-day survival and should become the focus of audit and quality improvement.

Future work

National audit of morbidities has been initiated. Further research is needed to understand the implications of feeding problems and renal failure and to evaluate the brief developmental assessment.

Funding

This project was funded by the NIHR Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 30. See the NIHR Journals Library website for further project information.

Identifying top 10 primary care research priorities from international stakeholders using a modified Delphi method

BACKGROUND

High quality primary care is fundamental to achieving health for all. Research priority setting is a key facilitator of improving how research activity responds to concrete needs. There has never before been an attempt to identify international primary care research priorities, in order to guide resource allocation and to enhance global primary care. This study aimed to identify a list of top 10 primary care research priorities, as identified by members of the public, health professionals working in primary care, researchers, and policymakers.

METHODS

We adapted the James Lind Alliance Priority Setting Partnership process, to conduct multiple rounds of stakeholder recruitment and prioritization. The study included an online survey conducted in three languages, followed by an in-person priority setting exercise involving primary care stakeholders from 13 countries.

FINDINGS

Participants identified a list of top 10 international primary care research priorities. These were focused on diverse topics such as enhancing use of information and communication technology, and improving integration of indigenous communities' knowledge in the design of primary care services. The main limitations of the study related to challenges in engaging an adequate diversity and number of appropriate stakeholders, particularly members of the public, in aggregating the diverse set of responses into coherent categories representative of the participants' perspectives and in adequately representing the diversity of submitted responses while ensuring research priorities on the final list are sufficiently actionable to guide resource allocation.

CONCLUSIONS

The top 10 identified research priorities have the potential to guide research resource allocation, supporting funding agencies and initiatives to promote global primary care research and practice.

Using a modified nominal group technique to develop general practice

Background

There are few areas of health care where sufficient research-based evidence exists and primary health care is no exception. In the absence of such evidence, the development of assisted support must be based on the opinions and experience of professionals with knowledge of the relevant field. The purpose of this research project is to explore how the nominal group technique can be used to establish consensus by analysing how it supported the development of structured, knowledge-based, electronic health records for preventive child health examinations in Danish general practice.

Methods

We convened an expert panel of five general practitioners with a special interest in the preventive child health examinations. We introduced the panel to the nominal group technique, a well-established, structured, multistep, facilitated, group meeting technique used to generate consensus. The panel used the technique to agree on the key clinical and socioeconomic themes to include in new electronic records for the seven preventive child health examinations in Denmark. The panel met three times over a four-month period between 2013 and 2014 and their meetings lasted between two-and-a-half and five hours.

Results

1) The structured and stepwise process of the nominal group technique supported our expert panel’s focus as well as their equal opportunities to speak. 2) The method’s flexibility enabled participants to work as a group and in pairs to discuss and refine thematic classifications. 3) Serial meetings supported continual evaluation, critical reflection, and knowledge searches, enabling our panel to produce a template that could be adapted for all seven preventive child health examinations.

Conclusion

The nominal group technique proved to be a useful method for reaching consensus by identifying key quality markers for use in daily clinical practice. Our study focused on the development of content and a layout for systematic, knowledge-based, electronic health records. We recommend the method as a suitable working tool for dealing with complex questions in general practice or similar settings, and we present and discuss modifications to the original model.

Definition of important early morbidities related to paediatric cardiac surgery

BACKGROUND

Morbidity is defined as a state of being unhealthy or of experiencing an aspect of health that is "generally bad for you", and postoperative morbidity linked to paediatric cardiac surgery encompasses a range of conditions that may impact the patient and are potential targets for quality assurance.

METHODS

As part of a wider study, a multi-disciplinary group of professionals aimed to define a list of morbidities linked to paediatric cardiac surgery that was prioritised by a panel reflecting the views of both professionals from a range of disciplines and settings as well as parents and patients.

RESULTS

We present a set of definitions of morbidity for use in routine audit after paediatric cardiac surgery. These morbidities are ranked in priority order as acute neurological event, unplanned re-operation, feeding problems, the need for renal support, major adverse cardiac events or never events, extracorporeal life support, necrotising enterocolitis, surgical site of blood stream infection, and prolonged pleural effusion or chylothorax. It is recognised that more than one such morbidity may arise in the same patient and these are referred to as multiple morbidities, except in the case of extracorporeal life support, which is a stand-alone constellation of morbidity.

CONCLUSIONS

It is feasible to define a range of paediatric cardiac surgical morbidities for use in routine audit that reflects the priorities of both professionals and parents. The impact of these morbidities on the patient and family will be explored prospectively as part of a wider ongoing, multi-centre study.

Primary care quality indicators for children: measuring quality in UK general practice

Background

Child health care is an important part of the UK general practice workload; in 2009 children aged <15 years accounted for 10.9% of consultations. However, only 1.2% of the UK’s Quality and Outcomes Framework pay-for-performance incentive points relate specifically to children.

Aim

To improve the quality of care provided for children and adolescents by defining a set of quality indicators that reflect evidence-based national guidelines and are feasible to audit using routine computerised clinical records.

Design and setting

Multi-step consensus methodology in UK general practice.

Method

Four-step development process: selection of priority issues (applying nominal group methodology), systematic review of National Institute for Health and Care Excellence (NICE) and Scottish Intercollegiate Guidelines Network (SIGN) clinical guidelines, translation of guideline recommendations into quality indicators, and assessment of their validity and implementation feasibility (applying consensus methodology used in selecting QOF indicators).

Results

Of the 296 national guidelines published, 48 were potentially relevant to children in primary care, but only 123 of 1863 recommendations (6.6%) met selection criteria for translation into 56 potential quality indicators. A further 13 potential indicators were articulated after review of existing quality indicators and standards. Assessment of the validity and feasibility of implementation of these 69 candidate indicators by a clinical expert group identified 35 with median scores 8 on a 9-point Likert scale. However, only seven of the 35 achieved a GRADE rating >1 (were based on more than expert opinion).

Conclusion

Producing valid primary care quality indicators for children is feasible but difficult. These indicators require piloting before wide adoption but have the potential to raise the standard of primary care for all children.

General practitioners' views on quality markers for children in UK primary care: a qualitative study

BACKGROUND

Children make up about 20% of the UK population and caring for them is an important part of a general practitioner's (GP's) workload. However, the UK Quality Outcomes Framework (pay-for-performance system) largely ignores children - less than 3% of the quality markers relate to them. As no previous research has investigated whether GPs would support or oppose the introduction of child-specific quality markers, we sought their views on this important question.

METHODS

Qualitative interview study with 20 GPs from four primary care trusts in Thames Valley, England. Semi-structured interviews explored GPs' viewpoints on quality markers and childhood conditions that could be developed into markers in general practice. Interviews were audiotaped and transcribed verbatim. Analysis was thematic and used constant comparative method to look for anticipated and emergent themes as the analysis progressed.

RESULTS

All the GPs interviewed supported the development of 'benchmarks' or 'standards' to measure and improve quality of care for children. However no consensus was expressed about the clinical conditions for which quality markers should be developed. Many participants reflected on their concerns about unmet health care needs and felt there may be opportunities to improve proactive care in 'at risk' groups. Some expressed feelings of powerlessness that important child-relevant outcomes such as emergency department visits and emergency admissions were out of their control and more directly related to public health, school and parents/carers. The importance of access was a recurrent theme; access to urgent general practice appointments for children and GP access to specialists when needed.

CONCLUSION

The GPs expressed support for the development of quality markers for the care of children in UK general practice. However, they flagged up a number of important challenges which need to be addressed if markers are to be developed that are measureable, targeted and within the direct control of primary care. Easy access to primary and secondary care appointments may be an important benchmark for commissioners of care.