A Patient Safety Toolkit for Family Practices

Automating detection of diagnostic error of infectious diseases using machine learning

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

Nurses' perceptions about the patient safety climate in Primary Health Care

OBJECTIVE

to evaluate the patient safety climate in Primary Health Care from the perspective of nurses working in the services.

METHOD

a quantitative and cross-sectional study conducted with 148 nurses from a municipality in the state of São Paulo. The Brazilian version of the Primary Care Safety Questionnaire Survey and personal, professional, and organizational performance variables (intention to stay at work, job satisfaction, care quality, and frequency of incidents) were used. Parametric and non-parametric comparison tests and Spearman's correlation coefficient were performed, considering a 5% significance level.

RESULTS

the safety climate was positive, varying from 4.52 to 5.33 and differing across districts for workload (p=0.0214) and leadership (p=0.0129). The safety climate professional variables and dimensions differed in relation to the frequency of incidents. Teamwork and safety and learning system were strongly correlated with job satisfaction and moderately with perceived care quality.

CONCLUSION

teamwork and safety and learning system stood out for their positive correlations with job satisfaction and care quality. A positive safety climate favors the involvement of Primary Care nurses to develop improvement plans aligned with the National Patient Safety Program.

BACKGROUND

(1) The safety climate is perceived differently across health districts. (2) There is a correlation between the climate dimensions and professional satisfaction. (3) Workload and leadership exert an influence on the safety climate perception. (4) There is a relationship between the safety climate and reporting of care-related incidents. (5) The safety climate is perceived differently among nurses regarding their role.

Comparative Analysis of the Impact of Training through Simulation Using the Crisis Resource Management Tool for Primary Care Professionals

This was a prospective observational study based on clinical simulation courses taught in 2017 at the IDEhA Simulation Center of Alcorcón Foundation University Hospital. Two courses in metabolic emergencies (MEs) and respiratory emergencies (REs) were offered to primary care physicians all over Spain. The main objective was to teach nontechnical skills (crisis resource management). Using a modified five-level Kirkpatrick-Phillips education evaluation model, level I (reaction, K1), level II (learning, K2) and level III (behavioral change, K3) changes were evaluated through surveys at the end of the courses and one year later. Thirty courses were held (15 ME courses and 15 RE courses) with 283 primary care physicians. The overall satisfaction (K1) was high: ME courses, 9.5/10; RE courses, 9.6/10. More than 80% of the participants rated the organization, resources, content, debriefing and scenarios as excellent, with no significant differences between the two courses. After one year (156 responses), the respondents for both courses reported that they would repeat the training annually (K2), encourage debriefing with colleagues (K3) and have modified some aspects of their workplace (K3), citing improvements in procedures and in the organization of the health team as the most important. After the ME course, few participants, i.e., 5 (6%), reported providing improved care to patients; after the RE course, 15 (19%) participants reported providing improved care; the difference between groups was significant (p < 0.05). Compared with the ME course, the RE course imparted greater knowledge about patient safety (K2) (38 (49%) vs. 24 (31%) (p < 0.05)) and more useful tools for daily clinical practice (K3) (67% vs. 56.4%) and resulted in participants paying more attention to personal performance and to colleagues when working as a team (K2) (64% vs. 50%). Clinical simulation courses are highly valued and potentially effective for training primary care physicians in patient safety and CRM tools. Future studies with objective measures of long-term impact, behavior in the workplace (K3) and benefits to patients (K4) are needed. Based on the results of our study, the areas that are important are those aimed at improving procedures and the organization of health teams.

Validation of a Reduced Set of High-Performance Triggers for Identifying Patient Safety Incidents with Harm in Primary Care: TriggerPrim Project

OBJECTIVE

The aim of the study was to construct and validate a reduced set of high-performance triggers for identifying adverse events (AEs) via electronic medical records (EMRs) review in primary care (PC).

METHODS

This was a cross-sectional descriptive study for validating a diagnostic test. The study included all 262 PC centers of Madrid region (Spain). Patients were older than 18 years who attended their PC center over the last quarter of 2018. The randomized sample was n = 1797. Main measurements were as follows: ( a ) presence of each of 19 specific computer-identified triggers in the EMR and ( b ) occurrence of an AE. To collect data, EMR review was conducted by 3 doctor-nurse teams. Triggers with statistically significant odds ratios for identifying AEs were selected for the final set after adjusting for age and sex using logistic regression.

RESULTS

The sensitivity (SS) and specificity (SP) for the selected triggers were: ≥3 appointments in a week at the PC center (SS = 32.3% [95% confidence interval {CI}, 22.8%-41.8%]; SP = 92.8% [95% CI, 91.6%-94.0%]); hospital admission (SS = 19.4% [95% CI, 11.4%-27.4%]; SP = 97.2% [95% CI, 96.4%-98.0%]); hospital emergency department visit (SS = 31.2% [95% CI, 21.8%-40.6%]; SP = 90.8% [95% CI, 89.4%-92.2%]); major opioids prescription (SS = 2.2% [95% CI, 0.0%-5.2%]; SP = 99.8% [95% CI, 99.6%-100%]); and chronic benzodiazepine treatment in patients 75 years or older (SS = 14.0% [95% CI, 6.9%-21.1%]; SP = 95.5% [95% CI, 94.5%-96.5%]).The following values were obtained in the validation of this trigger set (the occurrence of at least one of these triggers in the EMR): SS = 60.2% (95% CI, 50.2%-70.1%), SP = 80.8% (95% CI, 78.8%-82.6%), positive predictive value = 14.6% (95% CI, 11.0%-18.1%), negative predictive value = 97.4% (95% CI, 96.5%-98.2%), positive likelihood ratio = 3.13 (95% CI, 2.3-4.2), and negative likelihood ratio = 0.49 (95% CI, 0.3-0.7).

CONCLUSIONS

The set containing the 5 selected triggers almost triples the efficiency of EMR review in detecting AEs. This suggests that this set is easily implementable and of great utility in risk-management practice.

Feasibility of the SINERGIAPS ("Sinergias entre profesionales y pacientes para una Atención Primaria Segura") intervention for improving patient safety in primary care

AIM

The primary aim was to examine the feasibility of intervention delivery and of trial procedures. Secondary aims were to study the intervention uptake; its acceptability and perceived utility; and its potential to improve safety culture and avoidable hospital admissions.

METHODS

We conducted a 3-month, single-arm feasibility study in 10 primary care (PC) centres in Spain. Centres received information regarding patients' experiences of safety (through the Patient Reported Experiences and Outcomes of Safety in Primary Care [PREOS-PC] questionnaire), and were instructed to plan safety improvements based on that feedback. We used a bespoke online tool to recruit PC professionals, collect patient feedback, and deliver it to the centres, and to collect outcome data (patient safety culture [Medical Office Survey on Patient Safety Culture, MOSPSC questionnaire]). We measured recruitment and follow-up rates and intervention uptake (based on the number of safety improvement plans registered). We conducted semistructured interviews with 9 professionals to explore the intervention acceptability and perceived utility.

RESULTS

Of 256 professionals invited, 120 (47%) agreed to participate, and 97 completed baseline and postintervention measures. Of 780 patients invited, 585 (75%) completed the PREOS-PC questionnaire. Five of 10 centres (50%) designed an improvement plan, providing 27 plans in total (range per centre, 1-14). The intervention was perceived as a novel strategy for improving safety, although the healthcare professionals identified several factors limiting its acceptability and utility: lack of feedback at the individual professional level; potentially unrepresentative sample of patients providing feedback; and number of educational materials deemed overwhelming.

DISCUSSION

It is feasible to deliver the proposed intervention so long as the identified limitations are addressed.

Cross-Cultural Adaptation, Validation, and Piloting of the Patient Reported Experiences and Outcomes of Safety in Primary Care Questionnaire for Its Use in Spain

OBJECTIVE

This study aimed to cross-culturally adapt, validate, and pilot the Patient Reported Experiences and Outcomes of Safety in Primary Care questionnaire for its use in Spain.

METHODS

After setting up an expert panel to determine its content validity, the questionnaire was translated and back-translated, and subjected to cognitive testing. The questionnaire was piloted in a cross-sectional study in 10 primary health care centers in Spain. Fifty patients per center completed the questionnaire while waiting for an appointment. We estimated (i) the acceptability of the questionnaire (response rate), (ii) scores distribution (floor and ceiling effects), (iii) internal consistency (Cronbach α), and (iv) construct validity (exploratory factor analyses and correlation between scales). To examine patients' evaluations of patient safety, we followed a mixed-methods approach: (i) statistical analyses at the scale and item levels based on responses to standardized items and (ii) qualitative content analysis based on responses to open-ended questions.

RESULTS

Complete data were collected from 493 patients (participation rate, 77%). A ceiling effect was observed for 3 scales ("safety problems," "harm severity," "harm needs"). The internal consistency was adequate (α > 0.7) for the majority of scales. Exploratory factor analysis and correlation between scales suggested an appropriate construct validity. Two hundred twenty-six (45.8%) respondents experienced at least 1 safety problem, and 109 (23.2%) reported harm in the previous 12 months.

CONCLUSIONS

The multidimensional primary health care patient safety instrument Patient Reported Experiences and Outcomes of Safety in Primary Care is now available for its use in Spain. Initial testing demonstrates its potential for use in primary care. Future developments will further address its use in actual clinical practice.

A comparative assessment of two tools designed to support patient safety culture in UK general practice

BACKGROUND

The NHS has recognised the importance of a high quality patient safety culture in the delivery of primary health care in the rapidly evolving environment of general practice. Two tools, PC-SafeQuest and MapSaf, were developed with the intention of assessing and improving patient safety culture in this setting. Both have been made widely available through their inclusion in the Royal College of General Practitioners' Patient Safety Toolkit and our work offerss a timely exploration of the tools to inform practice staff as to how each might be usefully applied and in which circumstances. Here we present a comparative analysis of their content, and describe the perspectives of staff on their design, outputs and the feasibility of their sustained use.

METHODS

We have used a content analysis to provide the context for the qualitative study of staff experiences of using the tools at a representative range of practices recruited from across the Midlands (UK). Data was collected through moderated focus groups using an identical topic guide.

RESULTS

A total of nine practices used the PC-SafeQuest tool and four the MapSaf tool. A total of 159 staff completed the PC-SafeQuest tool 52 of whom took part in the subsequent focus group discussions, and 25 staff completed the MapSaf tool all of whom contributed to the focus group discussions. PC-SafeQuest was perceived as quick and easy to use with direct questions pertinent to the work of GP practices providing useful quantitative insight into important areas of safety culture. Though MaPSaF was more logistically challenging, it created a forum for synchronous cross- practice discussions raising awareness of perceptions of safety culture across the practice team.

CONCLUSIONS

Both tools were able to promote reflective and reflexive practice either in individual staff members or across the broader practice team and the oversight they granted provided useful direction for senior staff looking to improve patient safety. Because PC SafeQuest can be easily disseminated and independently completed it is logistically suited to larger practice organisations, whereas the MapSaf tool lends itself to smaller practices where assembling staff in a single workshop is more readily achieved.

Developing a patient safety guide for primary care: A co-design approach involving patients, carers and clinicians

BACKGROUND

Patients and carers should be actively involved in patient safety and empowered to use person-centred approaches where they are asked to both identify safety concerns and partner in preventing them.

OBJECTIVES

The aim of this study was to co-design a patient safety guide for primary care (PSG-PC) to support patients and carers to address key patient safety questions and identify key points where they can make their care safer. The objectives were to i) identify when and how patients and carers can be involved in primary care patient safety, and ii) identify the relevant information to include in the PSG-PC.

DESIGN

An experience-based co-design approach.

SETTING AND PARTICIPANTS

We conducted three workshops with patients, carers, community pharmacists and general practitioners to develop and refine the PSG-PC.

RESULTS

Participants identified both explicit and implicit issues of primary care patient safety especially relating to technical and relational components of involving patients and carers. The importance of communication, understanding roles and responsibilities, and developing partnerships between patients and health-care providers were considered essential for actively involving patients in patient safety. Co-developing the PSG-PC provided insight to improve care to develop the PSG-PC.

DISCUSSION

The PSG-PC is the first guide to be developed for primary care, co-designed with patients, carers, general practitioners and pharmacists. The PSG-PC will support patients and carers to partner with health-care professionals to improve patient safety addressing international and national priorities to continuously improve patient safety.