South African clinical practice guidelines quality measured with complex and rapid appraisal instruments

Evaluation of clinical practice guideline quality: comparison of two appraisal tools

Objective

The tools used for critically appraising the quality of clinical practice guidelines are complex and not suitable for the busy end users. So rapid, effective and simple instruments are more preferred. The aim of this study is to compare two critical appraisal tools: iCAHE as a rapid instrument and AGREE II as a complex instrument on guideline quality assessment.

Material and Methods

The diabetes mellitus guidelines of the Scottish Intercollegiate Guidelines Network (SIGN), the National Institute for Health and Clinical Excellence (NICE), the International Diabetes Federation (IDF) and the Society of Endocrinology and Metabolism of Turkey (SEMT) were assessed separately by four appraisers using the iCAHE and AGREE II instruments. The mean iCAHE criteria scores and the total and domain AGREE II scores given by the four appraisers are presented for each guideline.

Results

No statistically significant difference was detected between the iCAHE scale scores of the guidelines evaluated (P = 0.063). The rank of the guidelines according to their average total iCAHE and AGREE II instrument scores was similar. The iCAHE mean scores of the guidelines were as follows: NICE, 92.85%; SIGN, 92.85%; IDF, 66.07% and SEMT, 73.21%. The AGREE II mean scores of the guidelines were as follows: NICE, 87.13%; SIGN, 78.25%; IDF, 53.44% and SEMT, 53.22%.

Conclusions

In addition to being a quality scale, the iCAHE checklist is easy, practical and short to implement. It also helps the users to understand the quality of the guideline in a shorter time. To increase the use of guidelines, it is important that users with little experience and time use the iCAHE scale as a rapid appraisal tool, but more studies are needed to decide the best appraisal tool.

Enhancing capacity for clinical practice guidelines in South Africa

INTRODUCTION

Use of good quality, evidence-informed and up-to-date clinical practice guidelines (CPGs) has the potential to impact health outcomes. This paper describes the development, implementation and evaluation of a dedicated CPG training course to address the training needs of CPG stakeholders in South Africa.

METHODS

We first reviewed the content and teaching strategies of existing CPG courses. This review consisted of a systematic review of teaching and learning strategies for guideline teams and a document review of existing courses offered by international guideline groups, universities and professional groups. We then strengthened an existing CPG course and evaluated it.

RESULTS

We found no studies on teaching and learning strategies for guideline teams. We identified six CPG courses being offered as full courses (part of a postgraduate degree program) by universities or as independent training for continuing professional education by professional groups. Contents focused on new guideline development. One course included alternative methods of guideline approaches such as contextualization and adaptation. The format varied from face-to-face sessions, to online sessions, group exercises and discussions, seminar format and project based activities. The revised CPG four-month long course that we implemented was designed to be pragmatic, reflective and contextually relevant. It used local guideline examples, authentic tasks, and an online forum for discussions and resources. It covered de novo CPG development, alternative methods of development (adopting, contextualising, adapting), and implementing CPGs. Course evaluation identified strengths and areas for improvement.

CONCLUSION

Dedicated capacity development has potential to positively influence CPG development and implementation.

A novel method of rapid appraisal of clinical practice guidelines for children with enuresis

BACKGROUND

Enuresis (bedwetting) is a common but variably managed pediatric condition. Despite an abundance of published documents which provide recommendations for clinical evaluation and management of enuresis, no formal appraisal of their methodological quality has been undertaken.

OBJECTIVE

The objective of the study is to evaluate the quality of current pediatric guidelines for enuresis (bedwetting) using a novel method of appraisal.

STUDY DESIGN

A comprehensive gray literature search was undertaken to identify guideline documents that provided recommendations for management of enuresis in children and adolescents. The search strategy included guideline databases, targeted websites, Google search engines, and MEDLINE. Guideline documents included clinical practice guidelines, consensus documents, position statements, and other clinical review documents. Each document underwent basic appraisal by two independent assessors using the International Centre for Allied Health Evidence (iCAHE) Guideline Quality Checklist. Those documents which (1) had an iCAHE quality score of ≥10; (2) used a systematic search strategy; and (3) linked evidence to their recommendations underwent further detailed appraisal using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument.

RESULTS

Eighteen documents were shortlisted for basic appraisal. The iCAHE highlighted a lack of information regarding underlying evidence and dates (mean score 36% and 41%, respectively). Only three documents met basic quality criteria and progressed to detailed appraisal using the AGREE II. These included guidelines produced by the Paediatric Society of New Zealand and National Clinical Guideline Centre and a position statement from the Canadian Paediatric Society. All three guidelines presented clear and unambiguous recommendations (mean score 80%). However, information regarding stakeholder involvement was lacking (mean score 50%).

CONCLUSIONS

Several guidelines exist for the evaluation and management of children with enuresis, but many lack appropriate methodological quality standards. The guideline produced by the National Clinical Guideline Centre achieved the highest quality rating and is recommended for future adaptation and implementation in relevant clinical settings.

Evaluation of all African clinical practice guidelines for hypertension: Quality and opportunities for improvement

RATIONALE, AIMS, AND OBJECTIVES

Good-quality clinical practice guidelines (CPGs) provide recommendations based on current best-evidence summaries. Hypertension is a prevalent noncommunicable disease in Africa, with disastrous sequelae (stroke, heart, and kidney disease). Its effective management relies on good quality, current, locally relevant evidence. This paper reports on an all African review of the guidance documents currently informing hypertension management.

METHODS

Attempts were made to contact 62 African countries for formal guidance documents used nationally to inform diagnosis and management of hypertension. Their quality was assessed by using Appraisal of Guidelines for Research & Evaluation (AGREE) II, scored by 2 independent reviewers. Differences in domain scores were compared between documents written prior to 2011 and 2011 onward. Findings were compared with earlier African CPG reviews.

RESULTS

Guidelines and protocols were provided by 26 countries. Six used country-specific stand-alone hypertension guidelines, and 10 used protocols embedded in Standard Treatment Guidelines for multiple conditions. Six used guidelines developed by the World Health Organization, and 4 indicated ad hoc use of international guidance (US, Portugal, and Brazil). Only 1 guidance document met CPG construction criteria, and none scored well on all AGREE domain scores. The lowest-scoring domain was rigour of development. There was no significant quality difference between pre-2011 and post-2011 guidance documents, and there were variable AGREE II scores for the same CPGs when comparing the African reviews.

CONCLUSIONS

The quality of hypertension guidance used by African nations could be improved. The need for so many guidance documents is questioned. Adopting a common evidence base from international good-quality CPGs and layering it with local contexts offer 1 way to efficiently improve African hypertension CPG quality and implementation.

Quality and utilization of the Finnish clinical practice guideline in schizophrenia: evaluation using AGREE II and the vignette approach

PURPOSE

This study aims to evaluate the quality of the Finnish clinical practice guideline related to schizophrenia care and how it can be utilized in psychiatric services.

PARTICIPANTS AND METHODS

The data were collected from one psychiatric ward and seven psychiatric outpatient units situated in two cities in Southern Finland. A total of 49 professionals working in these sites participated in this study. A descriptive study design was adopted. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument with six domains including 23 items was used to assess the quality of the guideline, whereas a vignette was implemented to describe how staff would use the guideline in hypothetical cases in daily practice. The analysis of AGREE II was based on rating each of the 23 items on a seven-point scale and calculating a quality score for the six domains. To describe the utilization of the guideline, eight key recommendations of the guideline were deductively rated out of the respondents' answers.

RESULTS

The results showed that the "scope and purpose" of the guideline were well described, but "applicability" was insufficient. The overall quality of the guideline was high (73%). Almost one fifth of the respondents were in agreement with key recommendations.

CONCLUSION

The overall quality of Guideline for Schizophrenia was good, but its "applicability" and utilization should be improved.

National stakeholders' perceptions of the processes that inform the development of national clinical practice guidelines for primary healthcare in South Africa

Background

There is increased international focus on improving the rigour of clinical practice guideline (CPG) development practices. However, few empirical studies on CPG development have been conducted in low- and middle-income countries. This paper explores national stakeholders’ perceptions of processes informing CPG development for primary healthcare in South Africa, focusing on both their aspirations and views of what is actually occurring.

Methods

A qualitative study design was employed including individual interviews with 37 South African primary care CPG development role-players. Participants represented various disciplines, sectors and provinces. The data were analysed through thematic analysis and an interpretivist conceptual framework.

Results

Strongly reflecting current international standards, participants identified six ‘aspirational’ processes that they thought should inform South African CPG development, as follows: (1) evidence; (2) stakeholder consultation; (3) transparency; (4) management of interests; (5) communication/co-ordination between CPG development groups; and (6) fit-for-context. While perceptions of a transition towards more robust processes was common, CPG development was seen to face ongoing challenges with regards to all six aspirational processes. Many challenges were attributed to inadequate financial and human resources, which were perceived to hinder capacity to undertake the necessary methodological work, respond to stakeholders’ feedback, and document and share decision-making processes. Challenges were also linked to a complex web of politics, power and interests. The CPG development arena was described as saturated with personal and financial interests, groups competing for authority over specific territories and unequal power dynamics which favour those with the time, resources and authority to make contributions. These were all perceived to affect efforts for transparency, collaboration and inclusivity in CPG development.

Conclusion

While there is strong commitment amongst national stakeholders to advance CPG development processes, a mix of values, politics, power and capacity constraints pose significant challenges. Contrasting perspectives regarding managing interests and how best to adapt to within-country contexts requires further exploration. Dedicated resources for CPG development, standardised systems for managing conflicting interests, and the development of a political environment that fosters collaboration and more equitable inclusion within and between CPG development groups are needed. These initiatives may enhance CPG quality and acceptability, with associated positive impact on patient care.

South African primary health care allied health clinical practice guidelines: the big picture

BACKGROUND

Good quality clinical practice guidelines (CPGs) are a vehicle to implementing evidence into allied health (AH) care. This paper reports on the current 'state of play' of CPGs in a lower-to-middle-income country (South Africa), where primary healthcare (PHC) AH activities face significant challenges in terms of ensuring quality service delivery in the face of huge PHC need.

METHODS

A qualitative study was conducted, using semi-structured interviews with purposively-sampled individuals involved in AH PHC CPGs in South Africa. They included national and state government policy-makers, academics and educators, service managers, clinicians, representatives of professional associations, technical writers, and members of informal professional networks. The interview data was transcribed and de-identified, and analysed descriptively by hand-coding. The COREQ statement guided study conduct and reporting. A framework to guide research in other countries into perspectives of AH PHC CPG activities was established.

RESULTS

Of the 32 invited, 29 people participated: of these 25 were interviewed and four provided meeting notes. Most participants had multiple professional roles, being engaged concurrently in clinical practice, academia, professional associations and / or government. Key themes comprised Players (sub-themes of sampling frame, participants, advice, role players and collaboration); Guidance (sub-themes of nomenclature, drivers, purpose, evidence sources) and Role of AH in PHC (sub-themes of discipline groupings, disability and rehabilitation, AH recognition).

CONCLUSION

There was consistently-expressed desire for quality guidance to support better quality AH PHC activities around the country. However no international CPGs were used, and there were no South African CPGs specific to local PHC AH practice. The guidance gap was filled by non-evidence-based documents produced often without training, to deal with specific clinical situations. This led to frustration, duplication and fragmentation of effort, confusing nomenclature, and an urgent need for standardised and agreed guidance. We provided a standardised framework to capture perspectives on CPGs activities in other AH PHC settings.

Barriers and enablers for the development and implementation of allied health clinical practice guidelines in South African primary healthcare settings: a qualitative study

Background

The South African allied health (AH) primary healthcare (PHC) workforce is challenged with the complex rehabilitation needs of escalating patient numbers. The application of evidence-based care using clinical practice guidelines (CPGs) is one way to make efficient and effective use of resources. Although CPGs are common for AH in high-income countries, there is limited understanding of how to do this in low- to middle-income countries. This paper describes barriers and enablers for AH CPG uptake in South African PHC.

Methods

Semi-structured individual interviews were undertaken with 25 South African AH managers, policymakers, clinicians and academics to explore perspectives on CPGs. Interviews were conducted by researcher dyads, one being familiar with South African AH PHC practice and the other with CPG expertise. Rigour and transparency of data collection was ensured. Interview transcripts were analysed by structuring content into codes, categories and themes. Exemplar quotations were extracted to support themes.

Results

CPGs were generally perceived to be relevant to assist AH providers to address the challenges of consistently providing evidence-based care in South African PHC settings. CPGs were considered to be tools for managing clinical, social and economic complexities of AH PHC practice, particularly if CPG recommendations were contextusalised. CPG uptake was one way to deal with increasing pressures to make efficient use of scarce financial resources, and to demonstrate professional legitimacy. Themes comprised organisational infrastructures and capacities for CPG uptake, interactions between AH actors and interaction with broader political structures, the nature of AH evidence in CPGs, and effectively implementing CPGs into practice.

Conclusion

CPGs contextualised to local circumstances offer South African PHC AH services with an efficient vehicle for putting evidence into practice. There are challenges to doing this, related to local barriers such as geography, AH training, workforce availability, scarce resources, an escalating number of patients requiring complex rehabilitation, and local knowledge. Concerted attempts to implement locally relevant CPGs for AH primary care in South Africa are required to improve widespread commitment to evidence-based care, as well as to plan efficient and effective service delivery models.

A critical analysis of Australian policies and guidelines for water immersion during labour and birth

BACKGROUND

Accessibility of water immersion for labour and/or birth is often dependent on the care provider and also the policies/guidelines that underpin practice. With little high quality research about the safety and practicality of water immersion, particularly for birth, policies/guidelines informing the practice may lack the evidence necessary to ensure practitioner confidence surrounding the option thereby limiting accessibility and women's autonomy.

AIM

The aims of the study were to determine how water immersion policies and/or guidelines are informed, who interprets the evidence to inform policies/guidelines and to what extent the policy/guideline facilitates the option for labour and birth.

METHOD

Phase one of a three-phase mixed-methods study critically analysed 25 Australian water immersion policies/guidelines using critical discourse analysis.

FINDINGS

Policies/guidelines pertaining to the practice of water immersion reflect subjective opinions and views of the current literature base in favour of the risk-focused obstetric and biomedical discursive practices. Written with hegemonic influence, policies and guidelines impact on the autonomy of both women and practitioners.

CONCLUSION

Policies and guidelines pertaining to water immersion, particularly for birth reflect opinion and varied interpretations of the current literature base. A degree of hegemonic influence was noted prompting recommendations for future maternity care policy and guidelines'.

ETHICAL CONSIDERATIONS

The Human Research Ethics Committee of the University of South Australia approved the research.