An empirical test of short notice surveys in two accreditation programmes

Does advance contact with research participants increase response to questionnaires: an updated systematic review and meta-analysis

Background

Questionnaires remain one of the most common forms of data collection in epidemiology, psychology and other human-sciences. However, results can be badly affected by non-response. One way to potentially reduce non-response is by sending potential study participants advance communication. The last systematic review to examine the effect of questionnaire pre-notification on response is 10 years old, and lacked a risk of bias assessment.

Objectives

Update the section of the Cochrane systematic review, Edwards et al. (2009), on pre-notification to include 1) recently published studies, 2) an assessment of risk of bias, 3) Explore if heterogeneity is reduced by: delay between pre-contact and questionnaire delivery, the method of pre-contact, if pre-contact and questionnaire delivery differ, if the pre-contact includes a foot-in-the-door manipulation, and study’s the risk of bias.

Methods

Inclusion criteria: population: any population, intervention: comparison of some type of pre-notification, comparison group: no pre-notification, outcome: response rates. Study design: randomised controlled trails. Exclusion criteria: NA. Data sources: Studies which cited or were included in Edwards et al. (2009); We additionally searched: CINAHL, Web of Science, PsycInfo, MEDLINE, EconLit, EMBASE, Cochrane Central, Cochrane CMR, ERIC, and Sociological Abstracts. The searches were implemented in June 2018 and May 2021. Study screening: a single reviewer screened studies, with a random 10% sample independently screened to ascertain accuracy. Data extraction: data was extracted by a single reviewer twice, with a week between each extraction. Risk of Bias: within studies bias was assessed using the Cochrane Risk of Bias tool (ROB1) by a single unblinded reviewer, across studies bias was assessed using funnel plots. Synthesis Method: study results were meta-analysed with a random effects model using the final response rate as the outcome. Evaluation of Uncertainty: Uncertainty was evaluated using the GRADE approach.

Results

One hundred seven trials were included with 211,802 participants. Over-all pre-notification increased response, OR = 1.33 (95% CI: 1.20–1.47). However, there was a large amount of heterogeneity (I² = 97.1%), which was not explained by the subgroup analyses. In addition, when studies at high or unclear risk of bias were excluded the effect was to reduced OR = 1.09 (95% CI: 0.99–1.20). Because of the large amount of heterogeneity, even after restricting to low risk of bias studies, there is still moderate uncertainty in these results.

Conclusions

Using the GRADE evaluation, this review finds moderate evidence that pre-notification may not have an effect on response rates.

Funding

Economic and Social Research Council.

Preregistration

None.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01435-2.

A decade of commitment to hospital quality of care: overview of and perceptions on multicomponent quality improvement policies involving accreditation, public reporting, inspection and pay-for-performance

Background

Quality improvement (QI) initiatives such as accreditation, public reporting, inspection and pay-for-performance are increasingly being implemented globally. In Flanders, Belgium, a government policy for acute-care hospitals incorporates aforementioned initiatives. Currently, questions are raised on the sustainability of the present policy.

Objective

First, to summarise the various initiatives hospitals have adopted under government encouragement between 2008 and 2019. Second, to study the perspectives of healthcare stakeholders on current government policy.

Methods

In this multi-method study, we collected data on QI initiative implementation from governmental and institutional sources and through an online survey among hospital quality managers. We compiled an overview of QI initiative implementation for all Flemish acute-care hospitals between 2008 (n = 62) and 2019 (n = 53 after hospital mergers). Stakeholder perspectives were assessed via a second survey available to all healthcare employees and a focus group with healthcare policy experts was consulted. Variation between professions was assessed.

Results

QI initiatives have been increasingly implemented, especially from 2016 onwards, with the majority (87%) of hospitals having obtained a first accreditation label and all hospitals publicly reporting performance indicators, receiving regular inspections and having entered the pay-for-performance initiative. On the topic of external international accreditation, overall attitudes within the survey were predominantly neutral (36.2%), while 34.5% expressed positive and 29.3% negative views towards accreditation. In examining specific professional groups in-depth, we learned 58% of doctors regarded accreditation negatively, while doctors were judged to be the largest contributors to quality according to the majority of respondents.

Conclusions

Hospitals have demonstrated increased efforts into QI, especially since 2016, while perceptions on currently implemented QI initiatives among healthcare stakeholders are heterogeneous. To assure quality of care remains a top-priority for acute-care hospitals, we recommend a revision of the current multicomponent quality policy where the adoption of all initiatives is streamlined and co-created bottom-up.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07007-w.

Is a hospital quality policy based on a triad of accreditation, public reporting and inspection evidence-based? A narrative review

BACKGROUND

Since 2009, hospital quality policy in Flanders, Belgium, is built around a quality-of-care triad, which encompasses accreditation, public reporting (PR) and inspection. Policy makers are currently reflecting on the added value of this triad.

METHODS

We performed a narrative review of the literature published between 2009 and 2020 to examine the evidence base of the impact accreditation, PR and inspection, both individually and combined, has on patient processes and outcomes. The following patient outcomes were examined: mortality, length of stay, readmissions, patient satisfaction, adverse outcomes, failure to rescue, adherence to process measures and risk aversion. The impact of accreditation, PR and inspection on these outcomes was evaluated as either positive, neutral (i.e. no impact observed or mixed results reported) or negative.

OBJECTIVES

To assess the current evidence base on the impact of accreditation, PR and inspection on patient processes and outcomes.

RESULTS

We identified 69 studies, of which 40 were on accreditation, 24 on PR, three on inspection and two on accreditation and PR concomitantly. Identified studies reported primarily low-level evidence (level IV, n = 53) and were heterogeneous in terms of implemented programmes and patient populations (often narrow in PR research). Overall, a neutral categorization was determined in 30 articles for accreditation, 23 for PR and four for inspection. Ten of these recounted mixed results. For accreditation, a high number (n = 12) of positive research on adherence to process measures was discovered.

CONCLUSION

The individual impact of accreditation, PR and inspection, the core of Flemish hospital quality, was found to be limited on patient outcomes. Future studies should investigate the combined effect of multiple quality improvement strategies.

Using accreditation surveyors to conduct health services research: a qualitative, comparative study in Australia

OBJECTIVE

Healthcare accreditation surveyors are well positioned to gain access to hospitals and apply their existing data collection skills to research. Consequently, we contracted and trained a surveyor cohort to collect research data for the Deepening our Understanding of Quality in Australia (DUQuA) project. The aim of this study is to explore and compare surveyors' perceptions and experiences in collecting quality and safety data for accreditation and for health services research.

DESIGN

A qualitative, comparative study.

SETTING AND PARTICIPANTS

Ten surveyors participated in semi-structured interviews, which were audio recorded, transcribed and coded using Nvivo11. Interview transcripts of participants were analysed thematically and separately, providing an opportunity for comparison and for identifying common themes and subthemes.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Topics addressed data collection for healthcare accreditation and research, including preparation and training, structure, organization, attitudes and behaviours of staff and perceptions of their role.

RESULTS

Five themes and ten subthemes emerged from the interviews: (1) overlapping facilitators for accreditation and research data collection, (2) accreditation-specific facilitators, (3) overlapping barriers for accreditation and research data collection, (4) research data collection-specific barriers and (5) needs and recommendations. Subthemes were (1.1) preparation and training availability, (1.2) prior knowledge and experiences; (2.1) ease of access, (2.2) high staff engagement, (3.1) time, (4.1) poor access and structure, (4.2) lack of staff engagement, (4.3) organizational changes; (5.1) short-notice accreditation and (5.2) preparation for future research.

CONCLUSIONS

Although hospital accreditation and research activities require different approaches to data collection, we found that suitably trained accreditation surveyors were able to perform both activities effectively. The barriers surveyors encountered when collecting data for research provide insight into the challenges that may be faced when visiting hospitals for short-notice accreditation.

Mediators of change in healthcare organisations subject to external assessment: a systematic review with narrative synthesis

OBJECTIVES

External inspections are widely used to improve the quality of care. The effects of inspections remain unclear and little is known about how they may work. We conducted a narrative synthesis of research literature to identify mediators of change in healthcare organisations subject to external inspections.

METHODS

We performed a literature search (1980-January 2020) to identify empirical studies addressing change in healthcare organisations subject to external inspection. Guided by the Consolidated Framework for Implementation Research, we performed a narrative synthesis to identify mediators of change.

RESULTS

We included 95 studies. Accreditation was the most frequent type of inspection (n=68), followed by statutory inspections (n=19), and external peer review (n=9). Our findings suggest that the regulatory context in which the inspections take place affect how they are acted on by those being inspected. The way inspections are conducted seem to be critical for how the inspection findings are perceived and followed up. Inspections can engage and involve staff, facilitate leader engagement, improve communication and enable the creation of new networks for reflection on clinical practice. Inspections can contribute to creating an awareness of the inspected organisation's current practice and performance gaps, and a commitment to change. Moreover, they can contribute to facilitating the planning and implementation of change, as well as self-evaluation and the use of data to evaluate performance.

CONCLUSIONS

External inspections can affect different mediators of organisational change. The way and to what extent they do depend on a range of factors related to the outer setting, the way inspections are conducted and how they are perceived and acted on by the inspected organisation. To improve the quality of care, the organisational change processes need to involve and impact the way care is delivered to the patients.

Options to enhance the veracity of Australian health service accreditation assessments

Background:

Assessment processes applied within some health service accreditation programs have been criticised at times for being inaccurate, inconsistent or inefficient. Such criticism has inspired the development of innovative assessment methods.

Objective:

The Australian Commission on Safety and Quality in Health Care considered the use of three such methods: short-notice or unannounced methods; patient journey or tracer methods; and attestation by governing bodies.

Method:

A systematic search and synthesis of published peer-reviewed and grey literature associated with these methods.

Results and Conclusion:

The published literature demonstrates that the likely benefits of these three assessment methods warrant further evaluation, real-world trials and stakeholder consultation to determine the most appropriate models to introduce into national accreditation programs.

Implications:

The subsequent introduction of models of short-notice assessments and attestation by governing bodies into the Australian Health Service Safety and Quality Accreditation Scheme in January 2019 demonstrates how the findings presented in this article influenced the national change in assessment practice, providing an example of evidence-informed accreditation development.

Short-notice (48 hours) ACCREDITATION trial in Australia: stakeholder perception of assessment thoroughness, resource requirements and workforce engagement

Background

External, independent accreditation assessments of healthcare organisations are necessary to ensure the nationally legislated minimum standards of quality and safety (QS) are met. The predetermined scheduling of the assessments continues to be criticised due to the high level of organisational emphasis on preparing for accreditation.

Objectives

To determine the stakeholder perception of assessment thoroughness, staff resource requirements and workforce engagement changes if only 48 hours’ notice is given to an organisation prior to an accreditation assessment, compared with the standard-notice accreditation process.

Methods

Logan and Beaudesert Hospitals in Brisbane, Australia, trialled the ‘Short-Notice Survey Accreditation Assessment Process’ (SNAAP) between August 2017 and December 2018. The organisation was given just 48 hours’ notice prior to an accreditation assessment. Staff perception of the standard-notice accreditation process and short-notice process was assessed using a 5-point Likert scale repeated measures questionnaire (pretrial, 6 and 12 months after SNAAP launch).

Results

There was a statistically significant stakeholder opinion that SNAAP more effectively identified the true strengths and achievements of the organisation’s QS compared with ‘standard-notice’ survey (p=0.033). There was a significantly lower overall perceived proportion of staff resources required for SNAAP preparation in contrast to ‘standard-notice’ process (Baseline Av=21.38% vs Follow-up 1 and 2 Av=9.75%–6.25%, p=0.021). The questionnaire results reflected that SNAAP increased staff engagement in QS activities (Av=3.75 and 3.69, 95% CI=3.45–4.05 and 3.45–3.94).

Conclusions

With sufficient cultural and operational preparation to move to SNAAP, hospitals can potentially use SNAAP as a truer validation of QS standards, require less staffing resources to prepare for accreditation assessments and improve staff engagement in QS assurance and improvement.

Health service accreditation stimulating change in clinical care and human resource management processes: A study of 311 Australian hospitals

OBJECTIVE

This study aimed to establish whether longitudinal participation in an accreditation program is translated into improvement in continuity of quality patient care and human resource management (HRM) processes outcomes.

MATERIALS AND METHODS

This was a secondary data analysis of accreditation panel data from acute hospitals participating in the Australian Council on Healthcare Standards' Evaluation and Quality Improvement Program (EQuIP). EQuIP criteria data from 311 hospitals were collected by external surveyors across 2003-2006 (Time 1) and 2007-2010 (Time 2). Mandatory accreditation criteria ratings at Time 1 were used to determine hospital performance group membership (1 = below moderate, 2 = moderate, 3 = above moderate). Analysis was undertaken of ratings across continuity of quality patient care and HRM process criteria, at Time 1 and 2.

RESULTS

Continuity of quality patient care and HRM processes improved across time in the three performance groups. Lower performing hospitals improved at a greater rate than moderate and higher performing hospitals. The groupings and performance order did not change over time.

CONCLUSIONS

An accreditation program is an external driver that facilitates continual and systemic quality improvement changes to sub-systems with an organisation.

Impact of repeated hospital accreditation surveys on quality and reliability, an 8-year interrupted time series analysis

OBJECTIVE

To evaluate whether hospital re-accreditation improves quality, patient safety and reliability over three accreditation cycles by testing the accreditation life cycle model on quality measures.

DESIGN

The validity of the life cycle model was tested by calibrating interrupted time series (ITS) regression equations for 27 quality measures. The change in the variation of quality over the three accreditation cycles was evaluated using the Levene's test.

SETTING

A 650-bed tertiary academic hospital in Abu Dhabi, UAE.

PARTICIPANTS

Each month (over 96 months), a simple random sample of 10% of patient records was selected and audited resulting in a total of 388 800 observations from 14 500 records.

INTERVENTIONS

The impact of hospital accreditation on the 27 quality measures was observed for 96 months, 1-year preaccreditation (2007) and 3 years postaccreditation for each of the three accreditation cycles (2008, 2011 and 2014).

MAIN OUTCOME MEASURES

The life cycle model was evaluated by aggregating the data for 27 quality measures to produce a composite score (Y_C) and to fit an ITS regression equation to the unweighted monthly mean of the series.

RESULTS

The results provide some evidence for the validity of the four phases of the life cycle namely, the initiation phase, the presurvey phase, the postaccreditation slump and the stagnation phase. Furthermore, the life cycle model explains 87% of the variation in quality compliance measures (R²=0.87). The best-fit ITS model contains two significant variables (β₁ and β₃) (p≤0.001). The Levene's test (p≤0.05) demonstrated a significant reduction in variation of the quality measures (Y_C) with subsequent accreditation cycles.

CONCLUSION

The study demonstrates that accreditation has the capacity to sustain improvements over the accreditation cycle. The significant reduction in the variation of the quality measures (Y_C) with subsequent accreditation cycles indicates that accreditation supports the goal of high reliability.

Unannounced versus announced hospital surveys: a nationwide cluster-randomized controlled trial

Objective

To evaluate the effectiveness of unannounced versus announced surveys in detecting non-compliance with accreditation standards in public hospitals.

Design

A nationwide cluster-randomized controlled trial.

Setting and participants

All public hospitals in Denmark were invited. Twenty-three hospitals (77%) (3 university hospitals, 5 psychiatric hospitals and 15 general hospitals) agreed to participate.

Intervention

Twelve hospitals were randomized to receive unannounced surveys (intervention group) and eleven hospitals to receive announced surveys (control group). We hypothesized that the hospitals receiving the unannounced surveys would reveal a higher degree of non-compliance with accreditation standards than the hospitals receiving announced surveys. Nine surveyors trained and employed by the Danish Institute for Quality and Accreditation in Healthcare (IKAS) were randomized into teams and conducted all surveys.

Main outcome measure

The outcome was the surveyors’ assessment of the hospitals’ level of compliance with 113 performance indicators—an abbreviated set of the Danish Healthcare Quality Programme (DDKM) version 2, covering organizational standards, patient pathway standards and patient safety standards. Compliance with performance indicators was analyzed using binomial regression analysis with bootstrapped robust standard errors.

Results

In all, 16 202 measurements were acceptable for data analysis. The risk of observing non-compliance with performance indicators for the intervention group compared with the control group was statistically insignificant (risk difference (RD) = −0.6 percentage points [−2.51–1.31], P = 0.54). A converged analysis of the six patient safety critical standards, requiring 100% compliance to gain accreditation status revealed no statistically significant difference (RD = −0.78 percentage points [−4.01–2.44], P = 0.99).

Conclusions

Unannounced hospital surveys were not more effective than announced surveys in detecting quality problems in Danish hospitals.

Trial Registration number

ClinicalTrials.gov NCT02348567, https://clinicaltrials.gov/ct2/show/NCT02348567?term=NCT02348567.

The impact of national accreditation reform on survey reliability: a 2-year investigation of survey coordinators' perspectives

RATIONALE, AIMS AND OBJECTIVE

Accrediting health care organizations against standards is a recognized safety and quality intervention. The credibility of an accreditation programme relies on surveying reliability. We investigated accreditation survey coordinators' perceptions of reliability issues and their continued relevancy, during a period of national accreditation reform.

METHOD

In 2013 and 2014, questionnaire surveys were developed using survey coordinators' feedback of their experiences and concerns regarding the accreditation process. Each year, a purpose-designed questionnaire survey was administered during the accrediting agency survey coordinator training days.

RESULTS

Participants reported that survey reliability was informed by five categories of issues: the management of the accreditation process, including standards and health care organizational issues; surveyor workforce management; survey coordinator role; survey team; and individual surveyors. A new accreditation system and programme did not alter the factors reported to shape survey reliability. However, across the reform period, there was a noted change within each category of the specific issues that were of concern. Furthermore, consensus between coordinators that existed in 2013 appears to have diminished in 2014. Across all categories, in 2014 there was greater diversity of opinion than in 2013.

CONCLUSIONS

The known challenges to the reliability of an accreditation programme retained their potency and relevancy during a period of reform. The diversity of opinion identified across the coordinator workforce could potentially place the credibility and reliability of the new scheme at risk. The study highlights that reliability of an accreditation scheme is an ongoing achievement, not a one-off attainment.

The impact of Saudi hospital accreditation on quality of care: a mixed methods study

In 2005, the Central Board for Accreditation of Healthcare Institutions (CBAHI) was launched in Saudi Arabia in order to improve the quality of care. By 2010, the first hospital was accredited by CBAHI, followed by many hospitals in following years. The aim of this study is to examine the impact of CBAHI on quality of care. In this study we used a mixed methods approach involving surveys, documentary analyses and semi-structure interviews. Surveys data were collected from 669 staff. Documentary analyses included mortality, infection and length of stay. The semi-structure interview data were gathered from 12 senior managers. Data were collected from three accredited public hospitals. Although some improvements in procedure were recognised, CBAHI does not monitor the continuity of health care delivery and had no effect on quality outcomes in our analysis. This study illustrates a need to sustain improvements over time in the accreditation cycle. Copyright © 2016 John Wiley & Sons, Ltd.

Counting the costs of accreditation in acute care: an activity-based costing approach

OBJECTIVES

To assess the costs of hospital accreditation in Australia.

DESIGN

Mixed methods design incorporating: stakeholder analysis; survey design and implementation; activity-based costs analysis; and expert panel review.

SETTING

Acute care hospitals accredited by the Australian Council for Health Care Standards.

PARTICIPANTS

Six acute public hospitals across four States.

RESULTS

Accreditation costs varied from 0.03% to 0.60% of total hospital operating costs per year, averaged across the 4-year accreditation cycle. Relatively higher costs were associated with the surveys years and with smaller facilities. At a national level these costs translate to $A36.83 million, equivalent to 0.1% of acute public hospital recurrent expenditure in the 2012 fiscal year.

CONCLUSIONS

This is the first time accreditation costs have been independently evaluated across a wide range of hospitals and highlights the additional cost burden for smaller facilities. A better understanding of the costs allows policymakers to assess alternative accreditation and other quality improvement strategies, and understand their impact across a range of facilities. This methodology can be adapted to assess international accreditation programmes.

A hybrid health service accreditation program model incorporating mandated standards and continuous improvement: interview study of multiple stakeholders in Australian health care

The study aim was to investigate the understandings and concerns of stakeholders regarding the evolution of health service accreditation programs in Australia. Stakeholder representatives from programs in the primary, acute and aged care sectors participated in semi-structured interviews. Across 2011-12 there were 47 group and individual interviews involving 258 participants. Interviews lasted, on average, 1 h, and were digitally recorded and transcribed. Transcriptions were analysed using textual referencing software. Four significant issues were considered to have directed the evolution of accreditation programs: altering underlying program philosophies; shifting of program content focus and details; different surveying expectations and experiences and the influence of external contextual factors upon accreditation programs. Three accreditation program models were noted by participants: regulatory compliance; continuous quality improvement and a hybrid model, incorporating elements of these two. Respondents noted the compatibility or incommensurability of the first two models. Participation in a program was reportedly experienced as ranging on a survey continuum from "malicious compliance" to "performance audits" to "quality improvement journeys". Wider contextual factors, in particular, political and community expectations, and associated media reporting, were considered significant influences on the operation and evolution of programs. A hybrid accreditation model was noted to have evolved. The hybrid model promotes minimum standards and continuous quality improvement, through examining the structure and processes of organisations and the outcomes of care. The hybrid model appears to be directing organisational and professional attention to enhance their safety cultures. Copyright © 2015 John Wiley & Sons, Ltd.

A mechanism for revising accreditation standards: a study of the process, resources required and evaluation outcomes

Background

The study objective was to identify and describe the process, resources and expertise required for the revision of accreditation standards, and report outcomes arising from such activities.

Methods

Secondary document analysis of materials from an accreditation standards development agency. The Royal Australian College of General Practitioners’ (RACGP) documents, minutes and reports related to the revision of the accreditation standards were examined.

Results

The RACGP revision of the accreditation standards was conducted over a 12 month period and comprised six phases with multiple tasks, including: review methodology planning; review of the evidence base and each standard; new material development; constructing field trial methodology; drafting, trialling and refining new standards; and production of new standards. Over 100 individuals participated, with an additional 30 providing periodic input and feedback. Participants were drawn from healthcare professional associations, primary healthcare services, accreditation agencies, government agencies and public health organisations. Their expertise spanned: project management; standards development and writing; primary healthcare practice; quality and safety improvement methodologies; accreditation implementation and surveying; and research. The review and development process was shaped by five issues: project expectations; resource and time requirements; a collaborative approach; stakeholder engagement; and the product produced. The RACGP evaluation was that participants were positive about their experience, the standards produced and considered them relevant for the sector.

Conclusions

The revision of accreditation standards requires considerable resources and expertise, drawn from a broad range of stakeholders. Collaborative, inclusive processes that engage key stakeholders helps promote greater industry acceptance of the standards.

Health service accreditation reinforces a mindset of high-performance human resource management: lessons from an Australian study

OBJECTIVE

To investigate whether an accreditation program facilitates healthcare organizations (HCOs) to evolve and maintain high-performance human resource management (HRM) systems.

DESIGN

Cross-sectional multimethod study.

SETTING AND PARTICIPANTS

Healthcare organizations participating in the Australian Council on Healthcare Standards Evaluation and Quality Improvement Program (EQuIP 4) between 2007 and 2011.

MAIN OUTCOME MEASURES

Ratings across the EQuIP 4 HRM criteria, a clinical performance measure, surveyor reports (HRM information) and interview data (opinions and experiences regarding HRM and accreditation).

RESULTS

Healthcare organizations identified as high performing on accreditation HRM criteria seek excellence primarily because of internal motivations linked to best practice. Participation in an accreditation program is a secondary and less significant influence. Notwithstanding, the accreditation program provides the HCO opportunity for internal and external review and assessment of their performance; the accreditation activities are reflective learning and feedback events.

CONCLUSIONS

This study reveals that HCOs that pursue highly performing HRM systems use participation in an accreditation program as an opportunity. Their organizational mindset is to use the program as a tool by which to reflect and obtain feedback on their performance so to maintain or improve their management of staff and delivery of care.

Researching the reliability of accreditation survey teams: lessons learnt when things went awry

Accreditation of health organisations, occurring in over 70 countries, is predicated upon the reliability of survey teams judgements, but we do not know the extent to which survey teams are reliable. To contribute evidence to this issue, we investigated the reliability of two survey teams simultaneously assessing an organisation. The setting was a large Australian teaching hospital, and data were derived from interviews, observations and survey documents. Participants were from four groups: hospital staff, accreditation agency personnel and surveyors, and research staff. Thematic analysis was employed to identify significant factors that influenced the study. The two survey teams ratings and recommendations demonstrated high levels of agreement. However, while a common understanding of the study existed, the research was compromised. There were difficulties enacting the study. Contrary to negotiated arrangements, the pressure of the study resulted in surveyors discussing evidence and their interpretation of standards. Uncontrollable circumstances (late changes of personnel), and unexpected events (a breakdown of working relationships), challenged the study. The twin lessons learnt are that a consistent survey outcome is likely to be reached when reliability of process and consistent application of standards are pursued, and research requires negotiating challenges and relationships.

Stakeholder perspectives on implementing accreditation programs: a qualitative study of enabling factors

BACKGROUND

Accreditation programs are complex, system-wide quality and safety interventions. Despite their international popularity, evidence of their effectiveness is weak and contradictory. This may be due to variable implementation in different contexts. However, there is limited research that informs implementation strategies. We aimed to advance knowledge in this area by identifying factors that enable effective implementation of accreditation programs across different healthcare settings.

METHODS

We conducted 39 focus groups and eight interviews between 2011 and 2012, involving 258 diverse healthcare stakeholders from every Australian State and Territory. Interviews were semi-structured and focused on the aims, implementation and consequences of three prominent accreditation programs in the aged, primary and acute care sectors. Data were thematically analysed to distil and categorise facilitators of effective implementation.

RESULTS

Four factors were identified as critical enablers of effective implementation: the accreditation program is collaborative, valid and uses relevant standards; accreditation is favourably received by health professionals; healthcare organisations are capable of embracing accreditation; and accreditation is appropriately aligned with other regulatory initiatives and supported by relevant incentives.

CONCLUSIONS

Strategic implementation of accreditation programs should target the four factors emerging from this study, which may increase the likelihood of accreditation being implemented successfully.

The public disclosure of accreditation information in Australia: stakeholder perceptions of opportunities and challenges

Public disclosure is increasingly a requirement of accrediting agencies and governments. There are few published empirical evaluations of disclosure interventions that inform evidence-based implementation or policy. This study investigated the practices associated with the public disclosure of healthcare accreditation information, in addition to multi-stakeholder perceptions of key challenges and opportunities for improvement. We conducted a mixed methods study comprising analysis of disclosure practices by accreditation agencies, and 47 semi-structured individual or group interviews involving 258 people. Participants were diverse stakeholders associated with Australian primary, acute and residential aged care accreditation programmes. Four interrelated issues were identified. First, there was broad agreement that accreditation information should be publicly disclosed, although the three accreditation agencies differed in the information they made public. Second, two implementation issues emerged: the need to educate the community about accreditation information, and the practical question of the detail to be provided. Third, the impact, both positive and negative, of disclosing accreditation information was raised. Fourth, the lack of knowledge about the impact on consumers was discussed. Public disclosure of accreditation information is an idea that has widespread support. However, translating the idea into practice, so as to produce appropriate, meaningful information, is a challenge.

A multimethod research investigation of consumer involvement in Australian health service accreditation programmes: the ACCREDIT-SCI study protocol

INTRODUCTION

Health service accreditation programmes are a regulatory mechanism adopted to drive improvements inpatient safety and quality. Research investigating the benefits or limitations, of consumer involvement in accreditation programmes is negligible. To develop our knowledge in this area the ACCREDIT collaboration (Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork) has developed a research plan, known as the ACCREDIT-SCI (Standards of Consumer Involvement) study protocol. Two complementary studies have been designed: one, to examine the effectiveness of a standard for consumer participation and two, to explore how patient experiences vary across a range of settings with differing accreditation results.

METHODS AND DESIGN

The research setting is the Australian healthcare system, and the two studies focus on three accreditation programmes in the primary, acute and aged care domains. The studies will use multimethods: document analysis; interviews and surveys. Participants will be stakeholders across the three domains including: policy officers; frontline healthcare professionals; accreditation agency personnel, including surveyors and healthcare consumers. Drawing on previous experience, the research team has developed purpose-designed tools. Data will be analysed using thematic, narrative and statistical (descriptive and inferential) procedures.

ETHICS AND DISSEMINATION

The University of New South Wales Human Research Ethics Committee has approved the two studies (HREC 10274). Findings will be disseminated through seminars, conference presentations, academic publications and research partner websites. The findings will be formulated to facilitate uptake by policy and accreditation agency professionals, researchers and academics, and consumers, nationally and internationally.

Narrative synthesis of health service accreditation literature

AIMS

To systematically identify and synthesise health service accreditation literature.

METHODS

A systematic identification and narrative synthesis of health service accreditation literature published prior to 2012 were conducted. The search identified 122 empirical studies that examined either the processes or impacts of accreditation programmes. Study components were recorded, including: dates of publication; research settings; levels of study evidence and quality using established rating frameworks; and key results. A content analysis was conducted to determine the frequency of key themes and subthemes examined in the literature and identify knowledge-gaps requiring research attention.

RESULTS

The majority of studies (n=67) were published since 2006, occurred in the USA (n=60) and focused on acute care (n=79). Two thematic categories, that is, 'organisational impacts' and 'relationship to quality measures', were addressed 60 or more times in the literature. 'Financial impacts', 'consumer or patient satisfaction' and 'survey and surveyor issues' were each examined fewer than 15 times. The literature is limited in terms of the level of evidence and quality of studies, but highlights potential relationships among accreditation programmes, high quality organisational processes and safe clinical care.

CONCLUSIONS

Due to the limitations of the literature, it is not prudent to make strong claims about the effectiveness of health service accreditation. Nonetheless, several critical issues and knowledge-gaps were identified that may help stimulate and inform discussion among healthcare stakeholders. Ongoing effort is required to build upon the accreditation evidence-base by using high quality experimental study designs to examine the processes, effectiveness and financial value of accreditation programmes and their critical components in different healthcare domains.

Evaluation of current Australian health service accreditation processes (ACCREDIT-CAP): protocol for a mixed-method research project

INTRODUCTION

Accreditation programmes aim to improve the quality and safety of health services, and have been widely implemented. However, there is conflicting evidence regarding the outcomes of existing programmes. The Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork-Current Accreditation Processes (ACCREDIT-CAP) project is designed to address key gaps in the literature by evaluating the current processes of three accreditation programmes used across Australian acute, primary and aged care services.

METHODS AND DESIGN

The project comprises three mixed-method studies involving documentary analyses, surveys, focus groups and individual interviews. Study samples will comprise stakeholders from across the Australian healthcare system: accreditation agencies; federal and state government departments; consumer advocates; professional colleges and associations; and staff of acute, primary and aged care services. Sample sizes have been determined to ensure results allow robust conclusions. Qualitative information will be thematically analysed, supported by the use of textual grouping software. Quantitative data will be subjected to a variety of analytical procedures, including descriptive and comparative statistics. The results are designed to inform health system policy and planning decisions in Australia and internationally.

ETHICS AND DISSEMINATION

The project has been approved by the University of New South Wales Human Research Ethics Committee (approval number HREC 10274). Results will be reported to partner organisations, healthcare consumers and other stakeholders via peer-reviewed publications, conference and seminar presentations, and a publicly accessible website.