A qualitative analysis of a consensus process to develop quality indicators of injury care

An Examination of Perceptions among Black Women on Their Awareness of and Access to Pre-Exposure Prophylaxis (PrEP)

Cisgender Black women (CBW) experience 67% of new HIV diagnoses among women in the South. Progress toward ending the HIV epidemic requires researchers to explore perceptions of factors related to the decision to initiate pre-exposure prophylaxis (PrEP) among CBW. Qualitative methods were used to explore how social and structural constructs influence individual decisions to use PrEP among 20 CBW through focus groups. The thematic data analysis identified how facilitators and barriers to PrEP uptake aligned with an external locus of control (LOC) [e.g., media influences on understanding of PrEP] or an internal LOC (e.g., awareness of personal vulnerability to HIV). Several participants highlighted that their PrEP knowledge was rooted in an external LOC, such as media campaigns. A participant stated, 'But even with the commercial, it wasn't representation for me.' Another participant described her personal HIV vulnerability in her sexual relationship as an internal LOC, stating, 'Not ignorance, it's maybe just not accepting the true reality of this can be contracted even from someone that you believe that you trust.' Due to gaps in media marketing, healthcare providers should be aware that some female patients may perceive that PrEP is not for them. Provider-led sexual health discussions are urgently needed to bridge the gap between PrEP eligibility and initiation.

Parent-Child Grief Interactions: A Qualitative Analysis and Conceptual Framework of the Lived Experiences of Young Widowed Parents

A young parent's death is an unexpected event that incurs family stress and grief for the surviving parent and young children. However, few studies have examined widowed parents' grief experiences and parent-child interactions following a co-parent's death. Guided by phenomenology, this qualitative study examined the lived experiences of (N = 12) surviving parents grieving the loss of their co-parent. Data were collected using semi-structured interviews and analyzed using an inductive analytic procedure. Findings included themes of (1) not showing grief with child; (2) talking through grief/emotions with child; (3) maintaining connection between deceased parent and child; (4) timing of sharing things with children; and (5) utilizing bereavement and group support. These findings suggest that support services for surviving parents include providing information to them about the timing of when to share mementos with children and psychoeducation on emotion sharing and masking as part of the grief process with young children.

Developing a National Set of Health Equity Indicators Using a Consensus Building Process

BACKGROUND

Considerable health inequities documented in Israel between communities, populations and regions, undermine the rights of all citizens to optimal health. The first step towards health equity is agreement on a set of national indicators, reflecting equity in healthcare provision and health outcomes, and allowing monitoring of the impact of interventions on the reduction of disparities. We describe the process of reaching a consensus on a defined set of national equity indicators.

METHODS

The study was conducted between January 2019 and June 2020, in a multistage design: (A) Identifying appropriate and available inequity measures via interviews with stakeholders. (B) Agreement on the screening criteria (public health importance; gap characteristics; potential for change; public interest) and relative weighting. (C) Constructing the consultation framework as an online, 3-round Delphi technique, with a range of experts recruited from the health, welfare and education sectors.

RESULTS

Participants were of diverse age, gender, geographic location, religion and ethnicity, and came from academia, healthcare provision, government ministries and patient representative groups. Thirty measures of inequity, presented to participants, represented the following domains: Health promotion (11 indicators), acute and chronic morbidity (11), life expectancy and mortality (2), health infrastructures and affordability of care (4), education and employment (2). Of the 77 individuals contacted, 75 (97%) expressed willingness to participate, and 55 (73%) completed all three scoring rounds. The leading ten indicators were: Diabetes care, childhood obesity, adult obesity, distribution of healthcare personnel, fatal childhood injuries, cigarette smoking, infant mortality, ability to afford care, access to psychotherapy and distribution of hospital beds. Agreement among raters, measured as intra-class correlation coefficient (ICC), was 0.75.

CONCLUSION

A diverse range of consultants reached a consensus on the most important national equity indicators, including both clinical and system indicators. Results should be used to guide governmental decision-making and inter-sectoral strategies, furthering the pursuit of a more equitable healthcare system.

Multi-method versus single method appraisal of clinical quality indicators for the emergency medical services

BACKGROUND

Quality Indicator (QI) appraisal protocol is a novel methodology that combines multiple appraisal methods to comprehensively assess the 'appropriateness' of QIs for a particular healthcare setting. However, they remain inadequately explored compared to the single appraisal method approach.

OBJECTIVES

To describe and test a multi-method QI appraisal protocol versus the single method approach, against a series of QIs previously identified as potentially relevant to the prehospital emergency care setting.

METHODS

An appraisal protocol was developed consisting of two categorical-based appraisal methods, combined with the qualitative analysis of the discussion generated during the consensus application of each method. The output of the protocol was assessed and compared with the application and output of each method. Inter-rater reliability (IRR) of each particular method was evaluated prior to group consensus rating. Variation in the number of non-valid QIs and the proportion of non-valid QIs identified between each method and the protocol were compared and assessed.

RESULTS

There was mixed IRR of the individual methods. There was similarly low-to-moderate correlation of the results obtained between the particular methods (Spearman's rank correlation = 0.42, P < 0.001). From a series of 104 QIs, 11 non-valid QIs were identified that were shared between the individual methods. A further 19 non-valid QIs were identified and not shared by each method, highlighting the benefits of a multi-method approach. The outcomes were additionally evident in the group discussion analysis, which in and of itself added further input that would not have otherwise been captured by the individual methods alone.

CONCLUSION

The utilization of a multi-method appraisal protocol offers multiple benefits, when compared to the single appraisal approach, and can provide the confidence that the outcomes of the appraisal will ensure a strong foundation on which the QI framework can be successfully implemented.

Identifying the priority challenges in trauma care delivery for Australian and New Zealand trauma clinicians

INTRODUCTION

Injury is a leading cause of death and disability world-wide. Little is known about the day-to-day challenges the trauma clinicians face in their practice that they feel could be improved through an increased evidence base. This study explored and ranked the trauma clinical practice research priorities of trauma care professionals across Australia and New Zealand.

METHODS

A modified-Delphi study was conducted between September 2019 and January 2020. The study employed two rounds of online survey of trauma professionals from relevant Australia and New Zealand professional organisations using snowballing method. Participants were asked to rank the importance of 29 recommendations, each corresponding to a key challenge in trauma care delivery. Decisions on the priorities of the challenges were determined by a consensus of >70% of respondents ranking the challenge as important or very important.

RESULTS

One hundred and fifty-five participants completed Round One, and 106 participants completed Round Two. A total of 15 recommendations reached >70% in Round One. Nine recommendations also reached >70% consensus in Round Two. Recommendations ranked highest were 'Caring for elderly trauma patients', 'Identifying and validating key performance indicators for trauma system benchmarking and improvement', and 'Management of traumatic brain injury'.

CONCLUSION

This study identified the priority areas for trauma research as determined by clinician ranking of the most important for informing and improving their practice. Addressing these areas generates potential to improve the quality and safety of trauma care in Australian and New Zealand.

Content analysis of concussion education for coaches of youth and high school sport

OBJECTIVE

Describe the extent to which concussion education provided to coaches of US high school and youth sport is inclusive of information necessary for them to engage in target behaviors related to the prevention, identification and management of concussion.

METHODS

Sport organizations included in the sample were US state high school governing bodies and national governing bodies (NGBs) for youth sports with the highest concussion incidence rates. The concussion education resources made available online for coaches by these organizations were coded for content relevant to concussion prevention, identification and management.

RESULTS

All but one state high school governing body required coach concussion education, two NGBs required coach education, seven recommended it and only one did not require or recommend it. All educational materials covered at least 84% (n = 16/19) of the relevant knowledge domains.

DISCUSSION

Existing concussion education for coaches is addressing most, but not all, topics that may help coaches engage in concussion prevention, identification and management. Additional program development work is warranted to augment educational content related to coach communication about concussion safety with athletes and parents. Content appropriateness notwithstanding, further research is needed to understand the acceptability, efficacy, and implementation of concussion education for coaches.

Education, exposure and experience of prehospital teams as quality indicators in regional trauma systems

OBJECTIVE

Indicators to measure the quality of trauma care may be instrumental in benchmarking and improving trauma systems. This retrospective, observational study investigated whether data on three indicators for competencies of Dutch trauma teams (i.e. education, exposure, experience; agreed upon during a prior Delphi procedure) can be retrieved from existing registrations. The validity and distinctive power of these indicators were explored by analysing available data in four regions.

METHODS

Data of all polytrauma patients treated by the Helicopter Emergency Medical Services were collected retrospectively over a 1-year period. During the Delphi procedure, a polytrauma patient was defined as one with a Glasgow Coma Scale of 9 or less or a Paediatric Coma Scale of 9 or less, together with a Revised Trauma Score of 10 or less. Information on education, exposure and experience of the Helicopter Emergency Medical Services physician and nurse were registered for each patient contact.

RESULTS

Data on 442 polytrauma patients could be retrieved. Of these, according to the Delphi consensus, 220 were treated by a fully competent team (i.e. both the physician and the nurse fulfilled the three indicators for competency) and 22 patients were treated by a team not fulfilling all three indicators for competency. Across the four regions, patients were treated by teams with significant differences in competencies (P=0.002).

CONCLUSION

The quality indicators of education, exposure and experience of prehospital physicians and nurses can be measured reliably, have a high level of usability and have distinctive power.

Reconciling patient and provider priorities for improving the care of critically ill patients: A consensus method and qualitative analysis of decision making

Background

Providers have traditionally established priorities for quality improvement; however, patients and their family members have recently become involved in priority setting. Little is known about how to reconcile priorities of different stakeholder groups into a single prioritized list that is actionable for organizations.

Objective

To describe the decision‐making process for establishing consensus used by a diverse panel of stakeholders to reconcile two sets of quality improvement priorities (provider/decision maker priorities n=9; patient/family priorities n=19) into a single prioritized list.

Design

We employed a modified Delphi process with a diverse group of panellists to reconcile priorities for improving care of critically ill patients in the intensive care unit (ICU). Proceedings were audio‐recorded, transcribed and analysed using qualitative content analysis to explore the decision‐making process for establishing consensus.

Setting and participants

Nine panellists including three providers, three decision makers and three family members of previously critically ill patients.

Results

Panellists rated and revised 28 priorities over three rounds of review and reached consensus on the “Top 5” priorities for quality improvement: transition of patient care from ICU to hospital ward; family presence and effective communication; delirium screening and management; early mobilization; and transition of patient care between ICU providers. Four themes were identified as important for establishing consensus: storytelling (sharing personal experiences), amalgamating priorities (negotiating priority scope), considering evaluation criteria and having a priority champion.

Conclusions

Our study demonstrates the feasibility of incorporating families of patients into a multistakeholder prioritization exercise. The approach described can be used to guide consensus building and reconcile priorities of diverse stakeholder groups.

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study

OBJECTIVES

Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference.

DESIGN

Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference.

SETTING

A national health information steward, the Canadian Institute for Health Information (CIHI).

PARTICIPANTS

A total of 73 participants, comprised 61 conference attendants/stakeholders from across Canada and 12 national health information steward staff.

PRIMARY AND SECONDARY OUTCOME MEASURES

Indicator dispositions of retention, additional stakeholder consultation, further redevelopment or retirement.

RESULTS

4 dimensions (usability, importance, scientific soundness and feasibility) typically used to select measures for reporting were expanded to 18 criteria grouped under the 4 dimensions through a process of research and testing. Definitions for each criterion were developed and piloted. Once the definitions were established, 56 of CIHI's publicly reported HSP indicators were evaluated against the criteria using modified Delphi approaches. Of the 56 HSP indicators evaluated, 9 measures were ratified for retirement, 7 were identified for additional consultation and 3 for further research and development. A pre-Consensus Conference survey soliciting feedback from stakeholders on indicator recommendations received 48 responses (response rate of 79%).

CONCLUSIONS

A systematic evaluation of HSP indicators informed the development of objective recommendations for continued reporting. The evaluation was a fruitful exercise to identify technical considerations for calculating indicators, furthering our understanding of how measures are used by stakeholders, as well as harmonising actions that could be taken to ensure relevancy, reduce indicator chaos and build consensus with stakeholders.

Proposal of quality indicators for cardiac rehabilitation after acute coronary syndrome in Japan: a modified Delphi method and practice test

OBJECTIVES

Cardiac rehabilitation is underused and its quality in practice is unclear. A quality indicator is a measurable element of clinical practice performance. This study aimed to propose a set of quality indicators for cardiac rehabilitation following an acute coronary event in the Japanese population and conduct a small-size practice test to confirm feasibility and applicability of the indicators in real-world clinical practice.

DESIGN AND SETTING

This study used a modified Delphi technique (the RAND/UCLA appropriateness method), a consensus method which involves an evidence review, a face-to-face multidisciplinary panel meeting and repeated anonymous rating. Evidence to be reviewed included clinical practice guidelines available in English or Japanese and existing quality indicators. Performance of each indicator was assessed retrospectively using medical records at a university hospital in Japan.

PARTICIPANTS

10 professionals in cardiac rehabilitation for the consensus panel.

RESULTS

In the literature review, 23 clinical practice guidelines and 16 existing indicators were identified to generate potential indicators. Through the consensus-building process, a total of 30 indicators were assessed and finally 13 indicators were accepted. The practice test (n=39) revealed that 74% of patients underwent cardiac rehabilitation. Median performance of process measures was 93% (IQR 46-100%). 'Communication with the doctor who referred the patient to cardiac rehabilitation' and 'continuous participation in cardiac rehabilitation' had low performance (32% and 38%, respectively).

CONCLUSIONS

A modified Delphi technique identified a comprehensive set of quality indicators for cardiac rehabilitation. The single-site, small-size practice test confirmed that most of the proposed indicators were measurable in real-world clinical practice. However, some clinical processes which are not covered by national health insurance in Japan had low performance. Further studies will be needed to clarify and improve the quality of care in cardiac rehabilitation.

Development and pilot study of an essential set of indicators for general surgery services

INTRODUCTION

At present there is a lack of appropriate quality measures for benchmarking in general surgery units of Spanish National Health System. The aim of this study is to present the selection, development and pilot-testing of an initial set of surgical quality indicators for this purpose.

METHODS

A modified Delphi was performed with experts from the Spanish Surgeons Association in order to prioritize previously selected indicators. Then, a pilot study was carried out in a public hospital encompassing qualitative analysis of feasibility for prioritized indicators and an additional qualitative and quantitative three-rater reliability assessment for medical record-based indicators. Observed inter-rater agreement, prevalence adjusted and bias adjusted kappa and non-adjusted kappa were performed, using a systematic random sample (n=30) for each of these indicators.

RESULTS

Twelve out of 13 proposed indicators were feasible: 5 medical record-based indicators and 7 indicators based on administrative databases. From medical record-based indicators, 3 were reliable (observed agreement >95%, adjusted kappa index >0.6 or non-adjusted kappa index >0.6 for composites and its components) and 2 needed further refinement.

CONCLUSIONS

Currently, medical record-based indicators could be used for comparison purposes, whilst further research must be done for validation and risk-adjustment of outcome indicators from administrative databases. Compliance results in the adequacy of informed consent, diagnosis-to-treatment delay in colorectal cancer, and antibiotic prophylaxis show room for improvement in the pilot-tested hospital.

Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation

OBJECTIVE

In this article, we describe one approach for developing and evaluating quality indicators.

STUDY DESIGN AND SETTING

We focus on describing different conceptual approaches to quality indicator development, review one approach for developing quality indicators, outline how to evaluate quality indicators once developed, and discuss quality indicator maintenance.

RESULTS

The key steps for developing quality indicators include specifying a clear goal for the indicators; using methodologies to incorporate evidence, expertise, and patient perspectives; and considering contextual factors and logistics of implementation. The Strategic Framework Board and the National Quality Measure Clearinghouse have developed criteria for evaluating quality indicators that complement traditional psychometric evaluations. Optimal strategies for quality indicator maintenance and dissemination have not been determined, but experiences with clinical guideline maintenance may be informative.

CONCLUSION

For quality indicators to effectively guide quality improvement efforts, they must be developed, evaluated, maintained, and implemented using rigorous evidence-informed practices.