A Systematic Review of Primary Care Safety Climate Survey Instruments: Their Origins, Psychometric Properties, Quality, and Usage

Patient safety climate research in primary care dentistry: A systematic scoping review

BACKGROUND

Patient safety climate constitutes an important element for quality improvement. Its current evidence base has been generated in hospital settings in developed countries. Studies in dentistry are limited.

OBJECTIVE

To systematically explore the evidence regarding assessing patient safety climate in dentistry.

METHODS

We developed a search strategy to explore MEDLINE, SCOPUS, and Web of Science databases from January 1st, 2002, to December 31st, 2022, to include observational studies on patient safety culture or patient safety climate assessment. Methodological features and item data concerning the dimensions employed for assessment were extracted and thematically analyzed. Reported scores were also collected.

RESULTS

Nine articles out of 5584 were included in this study. Most studies were generated from high-income economies. Our analysis revealed methodological variations. Non-randomized samples were employed (ranging from 139 to 656 participants), and response rates varied from 28% to 93.7%. Three types of measurement instruments have been adapted to assess patient safety climate. These mainly consisted of replacing words or rewording sentences. Only one study employed an instrument previously validated through psychometric methods. In general, patient safety climate levels were either low or neutral. Only one study reported scores equal to or greater than 75.

DISCUSSION

Despite diverse assessment tools, our two-decade analysis reveals a lag compared with medicine, resulting in methodological variations for assessing patient safety climate. Collaboration is vital to elevate standards, prioritize patient safety across oral healthcare services, and advocate for integrating safety climate into local and national quality and patient safety strategies.

A Systematic Review of Patient-Report Safety Climate Measures in Health Care

OBJECTIVES

Patients are a valuable, yet underutilized source of information for safety measurement and improvement in health care. The aim of this review was to identify patient-report safety climate (SC) measures described in the literature, analyze the included items to consider their alignment with previously established SC domains, evaluate their validity and reliability, and make recommendations for best practice in using patient-report measures of SC in health care.

METHODS

Searches were conducted, with no limit on publication year, using MEDLINE, EMBASE, CINAHL, PsycINFO, and Academic Search Complete in November 2019. Reference lists of included studies and existing reviews were also screened. English-language, peer-reviewed studies that described the development or use of a patient-report measure to assess SC in health care were included. Two researchers independently extracted data from studies and applied a quality appraisal tool.

RESULTS

A total of 5060 studies were screened, with 44 included. Included studies described 31 different SC measures. There was much variability in the coverage of SC domains across included measures. Poor measure quality was marked by inadequacies in the testing and reporting of validity and reliability. There was also a lack of usability testing among measures.

CONCLUSIONS

This review identified the extant patient-reported SC measures in health care and demonstrated significant variance in their coverage of SC domains, validity and reliability, and usability. Findings suggest a pressing need for a stand-alone measure that has a high validity and reliability, and assess core SC domains from the patient perspective, particularly in primary care.

The adaptation of the 'Healthcare Complaints Analysis Tool' for general practice

BACKGROUND

Patient complaints about care in general practice are underutilized as a source of safety improvement data.

OBJECTIVE

This study aimed to adapt a secondary care complaints analysis tool for use in general practice contexts and assess the validity, reliability and usability of the adapted tool.

METHODS

The study was conducted in two phases. Phase A: The Healthcare Complaints Analysis Tool (HCAT) designed for use in secondary care was adapted for use in general practice using an iterative six-stage process. Phase B: Participants from key stakeholder groups [General practitioners (n = 5), complaints managers (n = 9), health service researchers (n = 4)]. Participants completed an online survey and analysed 20 fictionalized patient complaints using the adapted tool. Inter-rater reliability and agreement with a referent standard were analysed using Gwet's AC1 statistic.

RESULTS

Phase A: The HCAT was adapted to the Healthcare Complaints Analysis Tool (General Practice) [HCAT(GP)]. The HCAT(GP) tool consists of three domains (clinical, management and relationship problems), and seven categories. The HCAT(GP) had both content and face validity. Phase B: Inter-rater reliability was substantial for the HCAT(GP) categories (Gwet's AC1 = 0.65). Within-group agreement on the seven HCAT(GP) categories was substantial to perfect (AC1 0.61-0.85). Participants had substantial to perfect agreement with the referent standard across the survey with a mean AC1 of 0.899 (Range 0.76-0.97).

CONCLUSIONS

This study reports the adaptation of the HCAT(GP) and has established that the tool has sufficient validity, reliability and usability. This adapted tool can be applied to general practice complaints to identify areas for improvement.

Development and validation of a patient-report measure of safety climate for general practice

BACKGROUND

Patients remain an under-utilized source of information on patient safety, as reflected by the dearth of patient-report measures of safety climate, particularly for use in general practice settings. Extant measures are marked by poor coverage of safety climate domains, inadequate psychometric properties and/or lack of consideration of usability.

OBJECTIVE

To develop a novel patient-report measure of safety climate specifically for completion by general practice patients, and to establish the validity, reliability and usability of this measure.

METHODS

An iterative process was used to develop the safety climate measure, with patient and general practitioner input. A cross-sectional design was employed to examine the validity (content, construct and convergent), reliability (internal consistency), and usability (readability and burden) of the measure.

RESULTS

A total of 584 general practice patients completed the measure. The exploratory factor analysis identified five factors pertaining to safety climate in general practice: Feeling of Safety with GP; Practice Staff Efficiency and Teamwork; Staff Stress and Workload; Patient Knowledge and Accountability, and; Safety Systems and Behaviours. These factors strongly correlated with two global safety measures, demonstrating convergent validity. The measure showed strong internal consistency, and was considered usable for patients as indicated by readability and duration of completion.

CONCLUSION

Our novel measure of safety climate for use in general practice demonstrates favourable markers of validity, reliability and usability. This measure will provide a mechanism for the patient voice to be heard in patient safety measurement, and to be used to improve patient safety in general practice.

A meta-review of methods of measuring and monitoring safety in primary care

Background

A major barrier to safety improvement in primary care is a lack of safety data. The aims of this systematic meta-review (registration: CRD42021224367) were to identify systematic reviews of studies that examine methods of measuring and monitoring safety in primary care; classify the methods of measuring and monitoring safety in the included systematic reviews using the five safety domains of Vincent et al.’s framework and use this information to make recommendations for improving the measurement and monitoring of safety in primary care.

Methods

Four databases (Medline, Academic Search Complete, Web of Science and CINAHL) and the grey literature were screened in November 2020, with searches updated in January 2021. Systematic reviews were included if they addressed the measurement of patient safety in primary care and were published in English. Studies were assessed using the Critical Appraisal Skills Programme for systematic reviews.

Results

A total of 6904 papers were screened, with 13 systematic reviews included. A commonly reported method of measuring ‘past harm’ was through patient record review. The most frequent methods for assessing the ‘reliability of safety critical processes’ were checklists, observations and surveys of staff. Methods used to assess ‘sensitivity to operations’ included observation, staff surveys, interviews, focus groups, active monitoring and simulated patients. Safety climate surveys were a commonly used as an approach to assess ‘anticipation and preparedness’. A number of the reviews concluded that safety data could, and should, be used for ‘integration and learning’. The main limitation of the meta-review was that it was of systematic reviews only.

Conclusions

Many of the methods for measuring and monitoring safety are readily available, quick to administer, do not require external involvement and are inexpensive. However, there is still a need to improve the psychometric properties of many measures. Researchers must support the development of psychometrically sound safety measures that do not over burden primary care practitioners. Policymakers must consider how primary care practitioners can be supported to implement these measures.

Did healthcare professional perspectives on the quality and safety environment in New Zealand public hospitals change from 2012 to 2017?

PURPOSE

The work environment is known to influence professional attitudes toward quality and safety. This study sought to measure these attitudes amongst health professionals working in New Zealand District Health Boards (DHBs), initially in 2012 and again in 2017.

DESIGN/METHODOLOGY/APPROACH

Three questions were included in a national New Zealand health professional workforce survey conducted in 2012 and again in 2017. All registered health professionals employed with DHBs were invited to participate in an online survey. Areas of interest included teamwork amongst professionals; involvement of patients and families in efforts to improve patient care and ease of speaking up when a problem with patient care is perceived.

FINDINGS

In 2012, 57% of respondents (58% in 2017) agreed health professionals worked as a team; 71% respondents (73% in 2017) agreed health professionals involved patients and families in efforts to improve patient care and 69% (65% in 2017) agreed it was easy to speak up in their clinical area, with none of these changes being statistically significant. There were some response differences by respondent characteristics.

PRACTICAL IMPLICATIONS

With no change over time, there is a demand for improvement. Also for leadership in policy, management and amongst health professionals if goals of improving quality and safety are to be delivered upon.

ORIGINALITY/VALUE

This study provides a simple three-question method of probing perceptions of quality and safety and an important set of insights into progress in New Zealand DHBs.

Hospital Climate and Peer Report Intention on Adverse Medical Events: Role of Attribution and Rewards

Adverse medical events (AMEs) often occur in the healthcare workplace, and studies have shown that a positive atmosphere can reduce their incidence by increasing peer report intention. However, few studies have investigated the effect and action mechanism therein. We aimed to extend upon these studies by probing into the relationship between hospital climate and peer report intention, along with the mediating effect of attribution tendency and moderating effects of rewards. For this purpose, a cross-sectional survey was administered in a hospital among health professionals. We collected 503 valid questionnaires from health professionals in China and verified the hypothesis after sorting the questionnaires. The results of empirical analysis show that a positive hospital climate significantly induces individual internal attribution tendency, which in turn exerts a positive effect on peer report intention. Contract reward also helps to increase peer report intention, especially for health professionals with an internal attribution tendency. The findings contribute to the literature regarding AME management in hospitals by providing empirical evidence of the necessity for hospital climate and contract reward, and by providing insights to improve their integrated application.

Systematic review of interventions to improve gender equity in graduate medicine

Women are substantially underrepresented in senior and leadership positions in medicine and experience gendered challenges in their work settings. This systematic review aimed to synthesise research that has evaluated interventions for improving gender equity in medicine. English language electronic searches were conducted across MEDLINE, CINAHL, Academic Search Complete, PsycINFO and Web of Science. Reference list screening was also undertaken. Peer-reviewed studies published between 2000 and March 2020 that evaluated interventions to improve gender equity, or the experiences of women, in academic or clinical medicine were reviewed. Dual reviewer data extraction on setting, participants, type of intervention, measurement and outcomes was completed. Methodological rigour and strength of findings were evaluated. In total, 34 studies were included. Interventions were typically focused on equipping the woman (82.4%), that is, delivering professional development activities for women. Fewer focused on changing cultures (20.6%), ensuring equal opportunities (23.5%) or increasing the visibility or valuing of women (23.5%). Outcomes were largely positive (87.3%) but measurement typically relied on subjective, self-report data (69.1%). Few interventions were implemented in clinical settings (17.6%). Weak methodological rigour and a low strength of findings was observed. There has been a focus to-date on interventions which Equip the Woman Interventions addressing systems and culture change require further research consideration. However, institutions cannot wait on high quality research evidence to emerge to take action on gender equity. Data collated suggest a number of recommendations pertaining to research on, and the implementation of, interventions to improve gender equity in academic and clinical settings.

Quality assessment with diverse studies (QuADS): an appraisal tool for methodological and reporting quality in systematic reviews of mixed- or multi-method studies

BACKGROUND

In the context of the volume of mixed- and multi-methods studies in health services research, the present study sought to develop an appraisal tool to determine the methodological and reporting quality of such studies when included in systematic reviews. Evaluative evidence regarding the design and use of our existing Quality Assessment Tool for Studies with Diverse Designs (QATSDD) was synthesised to enhance and refine it for application across health services research.

METHODS

Secondary data were collected through a literature review of all articles identified using Google Scholar that had cited the QATSDD tool from its inception in 2012 to December 2019. First authors of all papers that had cited the QATSDD (n=197) were also invited to provide further evaluative data via a qualitative online survey. Evaluative findings from the survey and literature review were synthesised narratively and these data used to identify areas requiring refinement. The refined tool was subject to inter-rater reliability, face and content validity analyses.

RESULTS

Key limitations of the QATSDD tool identified related to a lack of clarity regarding scope of use of the tool and in the ease of application of criteria beyond experimental psychological research. The Quality Appraisal for Diverse Studies (QuADS) tool emerged as a revised tool to address the limitations of the QATSDD. The QuADS tool demonstrated substantial inter-rater reliability (k=0.66), face and content validity for application in systematic reviews with mixed, or multi-methods health services research.

CONCLUSION

Our findings highlight the perceived value of appraisal tools to determine the methodological and reporting quality of studies in reviews that include heterogeneous studies. The QuADS tool demonstrates strong reliability and ease of use for application to multi or mixed-methods health services research.

Using safety culture results to guide the merger of four general practices in the UK

BACKGROUND

The rapid merger in a crisis of three GP practices to incorporate the patients from a neighbouring closing surgery, led to the redesign of primary care provision. A deliberate focus on patient safety and staff engagement was maintained throughout this challenging transition to working at scale in an innovative, integrated and collaborative GP model.

METHOD

3 cycles of a staff culture tool (Safety, Communication, Organizational Reliability, Physician & Employee burn-out and Engagement) were performed at intervals of 9-12 months with structured feedback and engagement with staff after each round. The impact of different styles of feedback, the effect of specific interventions, and overall changes in safety climate and culture domains were observed in detail throughout this time period.

RESULTS

Strong themes demonstrated were that: there was a general improvement in all culture domains; specific focus on teams that expressed they were struggling created the most effective outcomes; an initial lack of trust of the management structure improved; adapting and tailoring the styles of feedback was most efficacious; and burn-out scores dropped progressively. A unique observation of the rate at which different modalities of safety climate and culture change with time is demonstrated.

CONCLUSION

With limited time, resources and energy, especially at times of crisis or change, the rapid and accurate identification of which domains of 'culture' and which teams required the most input at each stage of the journey is invaluable. Using this tool and prioritising patient safety, enables rapid and effective positive change to the culture and shape of expanding practices. It affirms that new models of working at scale in GP can be positively embraced with improvements in safety culture, if this is deliberately focused on and included in the transition process.

Barriers to Healthcare for Persons with Autism: A Systematic Review of the Literature and Development of A Taxonomy

Purpose: This review aimed to 1) synthesize extant research on barriers to healthcare access experienced by persons with autism, their caregivers, and health-care providers; and 2) present a taxonomy of barriers to physical healthcare for individuals with autism.Method: Systematic searches were conducted in five electronic databases. Methodological rigor was assessed using the Quality Assessment Tool for Studies with Diverse Designs. Thematic analysis was used to classify barriers and to develop a taxonomy.Results: In total, 31 articles were included in the review. The resulting taxonomy consisted of four themes: 1) Challenges Associated with Autism-related Characteristics; 2) Health-care Provider-based Issues; 3) Healthcare System Issues; and 4) Patient-related factors.Conclusions: Barriers to healthcare access for persons with autism are prevalent and occur at the patient, provider, and system levels. The taxonomy developed may facilitate measurement of barriers within health-care facilities and prompt identification of areas where interventions are warranted to improve care.

Hand Hygiene Compliance in the ICU: A Systematic Review

OBJECTIVES

To synthesize the literature describing compliance with World Health Organization hand hygiene guidelines in ICUs, to evaluate the quality of extant research, and to examine differences in compliance levels across geographical regions, ICU types, and healthcare worker groups, observation methods, and moments (indications) of hand hygiene.

DATA SOURCES

Electronic searches were conducted in August 2018 using Medline, CINAHL, PsycInfo, Embase, and Web of Science. Reference lists of included studies and related review articles were also screened.

STUDY SELECTION

English-language, peer-reviewed studies measuring hand hygiene compliance by healthcare workers in an ICU setting using direct observation guided by the World Health Organization's "Five Moments for Hand Hygiene," published since 2009, were included.

DATA EXTRACTION

Information was extracted on study location, research design, type of ICU, healthcare workers, measurement procedures, and compliance levels.

DATA SYNTHESIS

Sixty-one studies were included. Most were conducted in high-income countries (60.7%) and in adult ICUs (85.2%). Mean hand hygiene compliance was 59.6%. Compliance levels appeared to differ by geographic region (high-income countries 64.5%, low-income countries 9.1%), type of ICU (neonatal 67.0%, pediatric 41.2%, adult 58.2%), and type of healthcare worker (nursing staff 43.4%, physicians 32.6%, other staff 53.8%).

CONCLUSIONS

Mean hand hygiene compliance appears notably lower than international targets. The data collated may offer useful indicators for those evaluating, and seeking to improve, hand hygiene compliance in ICUs internationally.

Potential value of patient record review to assess and improve patient safety in general practice: A systematic review

Background: There is limited research, and guidance, on how to address safety in general practice proactively.

Objectives: This review aimed to synthesize the literature describing the use of patient record review (PRR) to measure and improve patient safety in primary care. The PRR methodologies utilized and the resulting outcomes were examined.

Methods: Searches were conducted using Medline, Embase, CINAHL and PsycINFO in February 2017. Reference lists of included studies and existing review papers were also screened. English language, peer-reviewed studies that utilized PRR to identify patient safety incidents (PSIs) occurring in general practice were included. Two researchers independently extracted data from articles and applied the Quality Assessment Tool for Studies with Diverse Designs.

Results: A total of 3265 studies were screened, with 15 included. Trigger tools were the most frequent method used for the PRRs (n = 6). The mean number of safety incidents per 100 records was 12.6. Within studies, a mean of 30.6% of incidents were associated with severe harm (range 8.6–50%), and a mean of 55.6% of incidents was considered preventable (range 32.7–93.5%). The most commonly identified types of PSIs related to medication and prescribing, diagnosis, communication and treatment. Three studies reported on improvement actions taken after the PRRs.

Conclusion: This review suggests that PRR may be a promising means of proactively identifying patient safety incidents and informing improvements.

Perceived safety climate in Irish primary care settings-a comparison with Scotland and England

Background: Safety climate (SC) measurement is a key component of quality and safety initiatives in primary healthcare.

Objectives: To (1) measure perceived SC in Irish primary care; (2) examine whether perceptions of safety varied according to respondent characteristics; and (3) compare responses from our Irish sample to data from England and Scotland.

Methods: PC-SafeQuest Survey was administered to all practice staff in Irish general practices between March and May 2016. This survey consists of 30 items across five safety domains (workload, communication, leadership, teamwork, and safety systems). Multiple regression analysis was used to identify predictor variables of perceived safety. The effect size of the difference between the Irish sample’s scores and published English and Scottish data were calculated.

Results: A total of 231 questionnaires (38.5%) were returned. Generally, positive perceptions of perceived safety were identified among Irish respondents, but workload had the lowest overall mean score (M = 4.3, SD = 1.2) of the five domains. Comparisons across the Irish, English and Scottish samples identified a medium size effect difference in workload; Scottish respondents perceived workload to have less of a negative impact on safety than Irish or English counterparts (Cohen’s d = 0.602, 0.67 respectively). Analyses indicated that Irish GP principals perceived a more negative impact of workload on safety than administrative staff (β = 0.28, P = 0.03).

Conclusion: Irish SC data are largely similar to those of England and Scotland. The perceived potential for workload to negatively impact upon safety emerged within each country. In Ireland, GP principals perceive this as a greater threat than practice administrators do.

Safety culture in the primary health care settings based on workers with a leadership role: the psychometric properties of the Slovenian-language version of the safety attitudes questionnaire - short form

Background

Safety culture describes leader and staff interactions, attitudes, routines, awareness, and practices within an organisation. With this study, we aimed to determine the psychometric properties of the Slovenian-language version of the Safety Attitudes Questionnaire (SAQ) – Short Form in primary health care settings.

Methods

This was a cross-sectional study in the largest primary health care in Slovenia. We invited all employees with a leadership role to participate in the study (N = 211). We used the Slovenian-language version of the SAQ – Short Form.

Results

There were 154 participants in the final sample (73.0% response rate), of which 136 (88.3%) were women. The mean age of the sample was 46.2 ± 10.0 years. Exploratory factor analysis put forward six factors: 1) Perceptions of Management; 2) Stress recognition; 3) Teamwork Climate; 4) Communication; 5) Safety Climate; 6) Working Conditions and Satisfaction. This model explained 61.7% of the variance of the safety culture in the primary health care setting. The reliability of the whole scale and of the six factors, assessed using Cronbach’s alpha, was all above 0.78.

Conclusion

The results of our study suggests that the Slovenian-language version of the SAQ – Short Form with six factors could be a reliable and valid tool for measuring the safety culture in the primary health care workers with leadership role In Slovenia. The Slovenian version differed from the original SAQ – Short Form and the majority of other translated versions. Also, the data was from one health centre only and therefore we cannot draw strong conclusions on its external validity.