Usability Checklists for Health Technology: Case Study and Experts' Opinions

Application of usability evaluations throughout the health technology lifecycle is necessary to improve the efficiency, safety, and effectiveness of health service delivery. Unfortunately, technology vendors and healthcare organizations may not have funding, time or expertise to conduct usability studies. In this paper, we describe how usability checklists can potentially fill this gap. First, we introduce a case study using a checklist to identify usability issues with a primary care dashboard. Then we provide an expert summary of the strengths and limitations of usability checklists. Findings suggest that checklists are efficient to identify important usability issues. They can be used effectively by project team members - including clinicians - without formal usability training. However, checklists should complement rather than replace usability evaluations with representative users.

A knowledge synthesis of health research reporting standards relevant to epilepsy surgery

OBJECTIVE

Numerous studies have examined epilepsy surgery outcomes, yet the variability in the level of detail reported hampers our ability to apply these findings broadly across patient groups. Established reporting standards in other clinical research fields enhance the quality and generalizability of results, ensuring that the insights gained from studying these surgeries can benefit future patients effectively. This study aims to assess current reporting standards for epilepsy surgery research and identify potential gaps and areas for enhancement.

METHODS

The Enhancing the Quality and Transparency of Health Research (EQUATOR) repository was accessed from inception to April 27, 2023, yielding 561 available reporting standards. Reporting standards were manually reviewed in duplicate independently for applicability to epilepsy and/or neurosurgery research. The reporting standards had to cover the following aspects in human studies: (1) reporting standards for epilepsy/epilepsy surgery and (2) reporting standards for neurosurgery. Disagreements were resolved by a third author. The top five neurosurgery, neurology, and medicine journals were also identified through Google Scholar's citation index and examined to determine the relevant reporting standards they recommended and whether those were registered with EQUATOR.

RESULTS

Of the 561 EQUATOR reporting standards, 181 were pertinent to epilepsy surgery. One was related to epilepsy, six were specific to surgical research, and nine were related to neurological/neurosurgical research. The remaining 165 reporting standards were applicable to research across various disciplines and included but were not limited to CONSORT (Consolidated Standards of Reporting Trails), STROBE (Strengthening the Reporting of Observational Studies in Epidemiology), and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). None of these required reporting factors associated with epilepsy surgery outcomes, such as duration of epilepsy or magnetic resonance imaging findings.

SIGNIFICANCE

Reporting standards specific to epilepsy surgery are lacking, reflecting a gap in standards that may affect the quality of publications. Improving this gap with a set of specific reporting standards would ensure that epilepsy surgery studies are more transparent and rigorous in their design.

Implementing culturally responsive and trauma-informed practices with checklists and goal setting

We discuss the use of two empirically validated behavior-change methods-checklists and goal setting-and designed a checklist to assist behavior analysts in improving their behavioral services to be more culturally responsive and trauma informed. We also present pilot data evaluating the use of the checklist and goal setting on the inclusion of culturally responsive and trauma-informed practices in behavior support plans designed for students in a public school. The training package was effective for both participants, and the participants' weekly goals corresponded to the observed changes in their behavior plans. Moreover, both participants strongly agreed that the checklist was valuable and easy to use and reported increases in their perceived abilities to implement culturally responsive and trauma-informed practices posttraining.

Ethical Checklists for Clinical Research Projects and laboratory medicine: two tools to evaluate compliance with bioethical principles in different settings

OBJECTIVES

To develop two ethical checklists to evaluate (i) management of ethical concerns in research projects and (ii) awareness of ethical conduct of healthcare laboratory professionals.

METHODS

Comprehensive discussion among the members of IFCC Task Force on Ethics based on pertinent literature.

RESULTS

This Checklist for Clinical Research Projects should be useful to evaluate research proposals from an ethical perspective before submitting it to an IRB or its equivalent, thereby diminishing rejection rates and resulting in more time-effective projects. The checklist designed to evaluate the ethical conduct in laboratory medicine could be useful for self evaluation (internal audits) and for certification/accreditation processes performed by third parties.

CONCLUSIONS

These checklists are simple but powerful tools useful to guide professionals to adhere to ethical principles in their practice, especially in developing countries where accredited ethics committees may be difficult to find.

Practical approaches for supporting informed consent in neonatal clinical trials

The survival and health of preterm and critically ill infants have markedly improved over the past 50  years, supported by well-conducted neonatal research. However, newborn research is difficult to undertake for many reasons, and obtaining informed consent for research in this population presents several unique ethical and logistical challenges. In this article, we explore methods to facilitate the consent process, including the role of checklists to support meaningful informed consent for neonatal clinical trials. CONCLUSION: The authors provide practical guidance on the design and implementation of an effective consent checklist tailored for use in neonatal clinical research.

Adaptation of the World Health Organization (WHO) Safe Surgery Checklist for Use With Cesarean Sections: Implementation and Outcomes With the Safe Cesarean Section Checklist

Introduction The World Health Organization (WHO) Safe Surgery Checklist significantly decreases morbidity and mortality in regular operating room cases. However, significant differences in workflow and processes exist between regular operating room cases and cesarean sections performed on the labor and delivery unit. The aim of this study is to adapt the WHO Safe Surgery Checklist for the labor and delivery unit and cesarean sections to improve communication and patient safety. Methods A multidisciplinary team consisting of all major stakeholders reviewed and revised the WHO Safe Surgery Checklist making it more applicable to cesarean section operations. The new Safe Cesarean Section Checklist was tested and then integrated into the electronic medical record and utilized on the labor and delivery unit. A specific cesarean section safety attitudes questionnaire was developed, validated, and administered prior to and one year after implementation. Results Usage of the Safe Cesarean Section Checklist was greater than 95% after initial implementation. Significant improvements were reported by the staff on the cesarean section attitudes questionnaire for several key areas including the feeling that all necessary information was available at the beginning of the procedure, decreases in communication breakdowns and delays, and fewer issues related to not knowing who was in charge during the procedure. Discussion Implementation of the Safe Cesarean Section Checklist was successfully adopted by the staff, and improvements in staff perceptions of several key safety issues on our unit were demonstrated. Additional studies should be undertaken to determine if clinical outcomes from this intervention are comparable to those seen with the use of the WHO Safe Surgery Checklist.

The Management of Intraoperative Spinal Cord Injury - A Scoping Review

STUDY DESIGN

Scoping Review.

OBJECTIVE

To review the literature and summarize information on checklists and algorithms for responding to intraoperative neuromonitoring (IONM) alerts and management of intraoperative spinal cord injuries (ISCIs).

METHODS

MEDLINE® was searched from inception through January 26, 2022 as were sources of grey literature. We attempted to obtain guidelines and/or consensus statements from the following sources: American Association of Neuromuscular & Electrodiagnostic Medicine (AANEM), American Academy of Neurology (AAN), American Clinical Neurophysiology Society, NASS (North American Spine Society), and other spine surgery organizations.

RESULTS

Of 16 studies reporting on management strategies for ISCIs, two were publications of consensus meetings which were conducted according to the Delphi method and eight were retrospective cohort studies. The remaining six studies were narrative reviews that proposed intraoperative checklists and management strategies for IONM alerts. Of note, 56% of included studies focused only on patients undergoing spinal deformity surgery. Intraoperative considerations and measures taken in the event of an ISCI are divided and reported in three categories of i) Anesthesiologic, ii) Neurophysiological/Technical, and iii) Surgical management strategies.

CONCLUSION

There is a paucity of literature on comparative effectiveness and harms of management strategies in response to an IONM alert and possible ISCI. There is a pressing need to develop a standardized checklist and care pathway to avoid and minimize the risk of postoperative neurologic sequelae.

Safe Surgery Checklist Implementation: Associations of Management Practice and Safety Culture Change

Designing and developing safe systems has been a persistent challenge in health care, and in surgical settings in particular. In efforts to promote safety, safety culture, i.e., shared values regarding safety management, is considered a key driver of high-quality, safe healthcare delivery. However, changing organizational culture so that it emphasizes and promotes safety is often an elusive goal. The Safe Surgery Checklist is an innovative tool for improving safety culture and surgical care safety, but evidence about Safe Surgery Checklist effectiveness is mixed. We examined the relationship between changes in management practices and changes in perceived safety culture during implementation of safe surgery checklists. Using a pre-posttest design and survey methods, we evaluated Safe Surgery Checklist implementation in a national sample of 42 general acute care hospitals in a leading hospital network. We measured perceived management practices among managers (n = 99) using the World Management Survey. We measured perceived preoperative safety and safety culture among clinical operating room personnel (N = 2,380 (2016); N = 1,433 (2017)) using the Safe Surgical Practice Survey. We collected data in two consecutive years. Multivariable linear regression analysis demonstrated a significant relationship between changes in management practices and overall safety culture and perceived teamwork following Safe Surgery Checklist implementation.

Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment

The integrity of randomized clinical trials (RCT) has become a concern owing to a recent rise in the number of retractions and the repercussions this has for evidence-based patient care. However, there is little research on the subject of RCT integrity assessment. Recent literature reviews have revealed that journals' authors' instructions concerning integrity and their investigation policies concerning allegations of misconduct are heterogeneous. The judicious use of integrity tests applied to RCT manuscripts is hampered by an absence of data concerning misconduct prevalence (pre-test probability), a failure to evaluate test performance (validity) and a lack of consensus over a gold standard (against which test accuracy can be evaluated). These deficiencies hinder the post-publication correction of RCT records, the integrity evaluations in systematic reviews of RCTs and the prospective application of preventive solutions in RCT peer-review and preprint assessment. Dealing with the current controversy about trustworthiness of RCT evidence requires a strong investment in research, reform and education concerning research integrity. The purpose of this review article is to highlight the current limitations in dealing with trial integrity-related concerns and to propose solutions to some of these issues.

Lessons from aviation safety: pilot monitoring, the sterile flight deck rule, and aviation-style computerised checklists in the operating room

Commercial aviation practices including the role of the pilot monitoring, the sterile flight deck rule, and computerised checklists have direct applicability to anaesthesia care. The pilot monitoring performs specific tasks that complement the pilot flying who is directly controlling the aircraft flight path. The anaesthesia care team, with two providers, can be organised in a manner that is analogous to the two-pilot flight deck. However, solo providers, such as solo pilots, can emulate the pilot monitoring role by reading checklists aloud, and utilise non-anaesthesia providers to fulfil some of the functions of pilot monitoring. The sterile flight deck rule states that flight crew members should not engage in any non-essential or distracting activity during critical phases of flight. The application of the sterile flight deck rule in anaesthesia practice entails deliberately minimising distractions during critical phases of anaesthesia care. Checklists are commonly used in the operating room, especially the World Health Organization surgical safety checklist. However, the use of aviation-style computerised checklists offers additional benefits. Here we discuss how these commercial aviation practices may be applied in the operating room.

Evaluating the uptake and effect of Surgical Safety Checklist implementation in a rural hospital, Neno District, Malawi

BACKGROUND

The WHO introduced the Surgical Safety Checklist (SSC) in 2008, which has been proven to enhance collaboration and patient safety before, during and after surgical procedures. However, the impact of using SSC has not been assessed in a rural setting in Malawi. We aimed to evaluate the uptake of SSC in Neno District, Malawi.

METHODS

We conducted a cross-sectional hospital-based retrospective chart review of 468 surgical cases from July 2021 to March 2022 in two hospitals in Neno District. We collected data using Excel and used R software for analysis. We used descriptive statistics to characterise the surgeries. We used χ2 test and Wilcoxon signed-rank test to test the association between SSC use and independent variables. We fitted logistic regression to assess predictors of SSC use and complications.

RESULTS

Of 468 surgical cases, 92% (n=431) were done as emergency procedures. The median age was 23 years (IQR: 19-29) and 94% (n=439) were female. Overall, 38% of surgeries (n=176) used the SSC and of these, 98% were in emergency procedures. We found an association between the use of SSC and the age of the client (p=0.018), type of procedure, name of the procedure, healthcare worker cadre, time procedure performed and complications (p<0.001). However, no association was observed between SSC use and outcome (p>0.05). The odds of using SSC were higher in emergency surgeries, surgeries performed at night; however, they were lower among temporary employees and anaesthetists with a diploma (p<0.001). The odds of experiencing complications were 1.71 times greater when using SSC compared with surgeries without SSC (p=0.029).

CONCLUSIONS

The use of an SSC in a rural area was promising, despite its fluctuating use and this needs programme improvement. Further studies are highly recommended to understand the fluctuation in the use of the SSC.

Sustainability and impact of the implementation of a frailty checklist for the acute medical unit: experience from a tertiary public hospital in Singapore

BACKGROUND

Accelerated population ageing is associated with an increasing prevalence of frailty. International guidelines call for systematic assessment and timely interventions for older persons requiring acute care. Checklists have been applied successfully in healthcare settings.

OBJECTIVE

This study describes the implementation of a safety checklist for frailty in the acute medical unit (AMU) of a tertiary public hospital in Singapore. We explored the sustainability of processes up to 6 months after initial implementation. Additionally, we investigated process and system outcome benefits following the implementation of the checklist.

METHODS

This retrospective observational study used case notes review of patients admitted to the AMU of a tertiary public hospital in Singapore from February to August 2019. Process outcomes measured to include compliance with AMU frailty checklist assessments and interventions at 24 hours of hospital admission. System and patient outcomes studied to include the length of hospital stay; 30-day emergency department reattendance rate; 30-day hospital readmission rate and inpatient mortality. Propensity scores were used to create balanced cohorts for comparison between those with complete and incomplete compliance with the checklist. Logistic regression was used to adjust for known confounders.

RESULTS

Average weekly (all-or-nothing) compliance with the frailty checklist (14.7%) was sustained for 6 months. Where assessments detected high risk, appropriate interventions were appropriately triggered (44%-97.4%). While trends to benefit systems and patient outcomes were present, these were not statistically significant. Contextual patterns are discussed.

CONCLUSION

A safety checklist for frailty was feasibly implemented in the AMU. The checklist was a complex intervention. Full compliance with the checklist was challenging to achieve. Further research assessing optimal patient selection criteria and how checklists may shift team behaviour is a priority.

Increasing the margin of patient safety for periodontal and implant treatments: The role of human factors

Early complications following periodontal and dental implant surgeries are typically attributed to technique or poor biological response, ignoring the possibility of the human element. Interestingly, significant experience is not correlated with increased success, whereas evidence supports the impact of clinical behavior on patient outcome. This is the result of errors, much like those scrutinized in other high-risk technical fields, such as aviation. What can be surprising is that those who make these errors are very well acquainted with best practices. Given this, how is it possible for the conscientious practitioner to fail to apply protocols that are nonetheless very well known? Recently, the concepts of human and organizational factors have been translated to medicine, though dentistry has been slow to recognize their potential benefit. This review lists specific human factor behaviors, such as use of checklists and crew resource management, which might improve postsurgical outcome.

Development and testing of the QDis-MH checklist for discharge letters from specialised mental healthcare: a stakeholder-centred study

BACKGROUND

The 'discharge letter' is the mandatory written report sent from specialists in the specialist services to general practitioners (GPs) on patient discharge. Clear recommendations from relevant stakeholders for contents of discharge letters and instruments to measure the quality of discharge letters in mental healthcare are needed. The objectives were to (1) detect which information relevant stakeholders defined as important to include in discharge letters from mental health specialist services, (2) develop a checklist to measure the quality of discharge letters and (3) test the psychometric properties of the checklist.

METHODS

We used a stepwise multimethod stakeholder-centred approach. Group interviews with GPs, mental health specialists and patient representatives defined 68 information items with 10 consensus-based thematic headings relevant to include in high-quality discharge letters. Information items rated as highly important by GPs (n=50) were included in the Quality of Discharge information-Mental Health (QDis-MH) checklist. The 26-item checklist was tested by GPs (n=18) and experts in healthcare improvement or health services research (n=15). Psychometric properties were assessed using estimates of intrascale consistency and linear mixed effects models. Inter-rater and test-retest reliability were assessed using Gwet's agreement coefficient (Gwet's AC1) and intraclass correlation coefficients.

RESULTS

The QDis-MH checklist had satisfactory intrascale consistency. Inter-rater reliability was poor to moderate, and test-retest reliability was moderate. In descriptive analyses, mean checklist scores were higher in the category of discharge letters defined as 'good' than in 'medium' or 'poor' letters, but differences did not reach statistical significance.

CONCLUSIONS

GPs, mental health specialists and patient representatives defined 26 information items relevant to include in discharge letters in mental healthcare. The QDis-MH checklist is valid and feasible. However, when using the checklist, raters should be trained and the number of raters kept to a minimum due to questionable inter-rater reliability.

Content validity of a Safe Nursing Care Checklist for a neonatal unit

BACKGROUND

Nursing checklists have been shown to improve communication, reduce the occurrence of adverse events, and promote safe, quality care in different care settings. However, to date, there is no validated patient care safety checklist for nurses caring for infants in Neonatal Intensive Care Units (NICU).

AIM

To describe development and content validation of the "Safe Nursing Care Checklist for Infants Hospitalized in the Neonatal Intensive Care Unit".

STUDY DESIGN

Online Survey.

METHODS

Based upon an integrative literature review, we developed a checklist focused on safe nursing care for infants in the NICU. Nursing experts participated in three rounds of a content validation process where they rated the items online. An agreement level ≥0.90 was required for inclusion in the checklist. Forty- three expert nurses with experience working in the NICU and who were certified in neonatal nursing or had a master's or doctoratal degree in child health provided content validation of the patient care checklist.

RESULTS

The final checklist contained 45 items with content validation index scores greater than 90%. The instrument was structured into six dimensions including patient identification, effective communication, medication safety, infection prevention, fall prevention, and pressure injuries/skin injuries prevention.

CONCLUSION

Content validity was established for the "Safe Nursing Care Checklist for Infants Hospitalized in the Neonatal Intensive Care Unit" which can identify strengths and weaknesses in safe nursing care for infants in the NICU as well as direct educational interventions to promote nursing care based on scientific evidence.

RELEVANCE TO CLINICAL PRACTICE

This checklist can potentially be used by bedside nurses to promote provision of safe care to infants in the NICU and to guide corrective strategies and encourage evidence-based decision-making. Validation in the clinical setting is needed.

Problem checklists and standardized diagnostic interviews: evidence of psychometric equivalence for classifying psychiatric disorder among children and youth in epidemiological studies

BACKGROUND

The standard approach for classifying child/youth psychiatric disorder as present or absent in epidemiological studies is lay-administered structured, standardized diagnostic interviews (interviews) based on categorical taxonomies such as the DSM and ICD. Converting problem checklist scale scores (checklists) to binary classifications provides a simple, inexpensive alternative.

METHODS

Using assessments obtained from 737 parents, we determine if child/youth behavioral, attentional, and emotional disorder classifications based on checklists are equivalent psychometrically to interview classifications. We test this hypothesis by (1) comparing their test-retest reliabilities based on kappa (κ), (2) estimating their observed agreement at times 1 and 2, and (3) in structural equation models, comparing their strength of association with clinical status and reported use of prescription medication to treat disorder. A confidence interval approach is used to determine if parameter differences lie within the smallest effect size of interest set at ±0.125.

RESULTS

The test-retest reliabilities (κ) for interviews compared with checklists met criteria for statistical equivalence: behavioral, .67 and .70; attentional, .64 and .66; and emotional, .61 and .65. Observed agreement between the checklist and interviews on classifications of disorder at times 1 and 2 was, on average, κ = .61. On average, the β coefficients estimating associations with clinical status were .59 (interviews) and .63 (checklists); and with prescription medication use, .69 (interviews) and .71 (checklists). Behavioral and attentional disorders met criteria for statistical equivalence. Emotional disorder did not, although the coefficients were stronger numerically for the checklist.

CONCLUSIONS

Classifications of child/youth psychiatric disorder from parent-reported checklists and interviews are equivalent psychometrically. The practical advantages of checklists over interviews for classifying disorder (lower administration cost and respondent burden) are enhanced by their ability to measure disorder dimensionally. Checklists provide an option to interviews in epidemiological studies of common child/youth psychiatric disorders.

Implementation of systematic safety checklists in a neurocritical care unit: a quality improvement study

BACKGROUND AND OBJECTIVES

Structured and systematised checklists have been shown to prevent complications and improve patient care. We evaluated the implementation of systematic safety checklists in our neurocritical care unit (NCCU) and assessed its effect on patient outcomes.

DESIGN/METHODS

This quality improvement project followed a Plan-Do-Study-Act (PDSA) methodology. A checklist for medication reconciliation, thromboembolic prophylaxis, glycaemic control, daily spontaneous awakening, breathing trial, diet, catheter/lines duration monitoring and antibiotics de-escalation was implemented during daily patient rounds. Main outcomes included the rate of new infections, mortality and NCCU-length of stay (LOS). Intervened patients were compared with historical controls after propensity score and Euclidean distance matching to balance baseline covariates.

RESULTS

After several PDSA iterations, we applied checklists to 411 patients; the overall average age was 61.34 (17.39). The main reason for admission included tumour resection (31.39%), ischaemic stroke (26.76%) and intracerebral haemorrhage (10.95%); the mean Sequential Organ Failure Assessment (SOFA) score was 2.58 (2.68). At the end of the study, the checklist compliance rate throughout the full NCCU stays reached 97.11%. After controlling for SOFA score, age, sex and primary admitting diagnosis, the implementation of systematic checklists significantly correlated with a reduced LOS (ß=-0.15, 95% CI -0.24 to -0.06), reduced rate of any new infections (OR 0.59, 95% CI 0.40 to 0.87) and reduced urinary tract infections (UTIs) (OR 0.23, 95% CI 0.09 to 0.55). Propensity score and Euclidean distance matching yielded 382 and 338 pairs with excellent covariate balance. After matching, outcomes remained significant.

DISCUSSION

The implementation of safety checklists in the NCCU proved feasible, easy to incorporate into the NCCU workflow, and a helpful tool to improve adherence to practice guidelines and quality of care measurements. Furthermore, our intervention resulted in a reduced NCCU-LOS, rate of new infections and rate of UTIs compared with propensity score and Euclidean distance matched historical controls.

A hotspot of lichen diversity and lichenological research in the Alps: the Paneveggio-Pale di San Martino Natural Park (Italy)

A checklist of 916 lichenised taxa is reported from the Paneveggio-Pale di San Martino Natural Park and its surroundings (Trentino-Alto Adige, N Italy), based on 7351 records from: (a) 72 literature sources, (b) eight public and private herbaria and (c) field observations by some of the authors. The study area appears as a hotspot of lichen diversity, hosting 30.1% of the lichen biota of the Alps in a territory that has 0.064% of their total surface area. This is mainly due to its high climatical, geological and orographic heterogeneity, but also to the long history of lichenological exploration, that started in the 19^th century with Ferdinand Arnold and is still ongoing. The present work highlights the importance of detailed species inventories to support knowledge of biodiversity patterns, taxonomy and ecology and to properly address conservation issues. Fuscideamollisvar.caesioalbescens, Hydropunctariascabra, Protoparmeliabadiavar.cinereobadia and Variosporapaulii are new to Italy, 18 other taxa are new to Trentino-Alto Adige.

New take on the post-take ward round: a quality improvement project undertaken in a district general hospital

A patient's first encounter with a consultant clinician, known as the post-take ward round (PTWR), is a pivotal encounter at the start of their hospital journey. It is a chance for a review of history, examination and investigations, formulation of preliminary diagnosis and management plan. High-quality patient care is reliant on effective communication of clinical information between teams, and the PTWR record is an integral part of this handover of information across different clinicians, medical teams and wards.Consensus of consultant opinion allowed for the formation of a standard against which the quality of PTWR documentation could be measured. This project aimed to assess and improve compliance with the devised standard.Following a survey of referrals made to the medical team after the move to electronic record keeping, it was found that important information was being missed from PTWR records. For example, of the 446 records analysed, only 34% had a documented potential discharge date (PDD) and 20% had a documented escalation plan. Analysis showed overall compliance to core criteria was 63%.Several changes within the department of acute medicine were trialled, including the introduction of a checklist, prompt cards for clinical staff to carry and finally the implementation of an electronic form for PTWR records.Over the course of several cycles of data collection, compliance with core criteria improved from 63% to 86%. Most notably, improvement was seen in documentation of social history (42%-87%), frailty score (0%-63%), PDD (41%-81%) and escalation plan (21%-66%).This work demonstrates the value of development of a standard for PTWR documentation, and of a proforma. The actions taken in this hospital may be of benefit to other medical departments.

Developing of Multidimensional Perspectives Checklist of Professionalism for Undergraduate Occupational Therapy Students in Assistive Technology Service: Delphi Study

Professionalism is a critical attribute that occupational therapy students must establish throughout education, especially in the context of assistive technology (AT). This study aimed to construct a multidimensional perspectives checklist (MPC) from the stakeholders to reflect professionalism development in the AT service courses. Fifteen experts in AT clinical practice and educators (N = 15) consented to and completed three rounds of the Delphi survey. Data were qualitatively analyzed to form a preliminary checklist in the first round. In rounds two and three, participants rated their level of agreement with the MPC items. A descriptive analysis of consensus was performed. Thirty items were classified into five subscales (teachers, therapists, patients, peers, learners) and fit into a framework with three dimensions and seven categories. After the Delphi survey, the MPC achieved high consensus, convergence, and stability. Two additional categories of professionalism emerged in the study, namely aesthetic and personal characteristics and reflection. The MPC developed in this study reflects the perspectives of various stakeholders in occupational therapy practice, providing helpful information for students to prepare themselves. Therefore, the MPC could contribute to expanding and developing the scope of professionalism in occupational therapy, especially in AT service.

Effect of the surgical safety checklist on provider and patient outcomes: a systematic review

BACKGROUND

Despite being implemented for over a decade, literature describing how the surgical safety checklist (SSC) is completed by operating room (OR) teams and how this relates to its effectiveness is scarce. This systematic review aimed to: (1) quantify how many studies reported SSC completion versus described how the SSC was completed; (2) evaluate the impact of the SSC on provider outcomes (Communication, case Understanding, Safety Culture, CUSC), patient outcomes (complications, mortality rates) and moderators of these relationships.

METHODS

A systematic literature search was conducted using Medline, CINAHL, Embase, PsycINFO, PubMed, Scopus and Web of Science on 10 January 2020. We included providers who treat human patients and completed any type of SSC in any OR or simulation centre. Statistical directional findings were extracted for provider and patient outcomes and key factors (eg, attentiveness) were used to determine moderating effects.

RESULTS

300 studies were included in the analysis comprising over 7 302 674 operations and 2 480 748 providers and patients. Thirty-eight per cent of studies provided at least some description of how the SSC was completed. Of the studies that described SSC completion, a clearer positive relationship was observed concerning the SSC's influence on provider outcomes (CUSC) compared with patient outcomes (complications and mortality), as well as related moderators.

CONCLUSION

There is a scarcity of research that examines how the SSC is completed and how this influences safety outcomes. Examining how a checklist is completed is critical for understanding why the checklist is successful in some instances and not others.

Checklists for Assessing Skills of Children With Type 1 Diabetes on Insulin Injection Technique

BACKGROUND

School-aged children often participate in type 1 diabetes (T1D) self-care tasks. Despite widespread discussion about the importance of developing self-care skills in childhood, few explain how the health care team should assess the skills of children with T1D when performing insulin injections.

OBJECTIVE

We sought to assess content validity evidence in two checklists regarding injection technique performed by children.

METHODS

Two checklists were designed based on a systematic review of the insulin injection technique. Experts in pediatric diabetes, health literacy, and diabetes education assessed the checklists regarding their clarity, objectivity, and relevance. Content validity was assessed using the content validity ratio (CVR).

RESULTS

Eleven providers (72% nurses or physicians, professional experience 19.4 ± 10.1 years, 45% of specialists in endocrinology, and 18% in pediatrics) participated in the assessment. Experts considered items containing the word homogeneity inappropriate. Items related to the needle insertion angle and the skin fold did not reach the CVR critical value. The final version of the checklist for syringe injection comprised 22 items with CVR = 0.91, and the checklist for pen injection comprised 18 items with CVR = 0.87.

CONCLUSIONS

The checklists presented clear, objective, and relevant content that assesses the skills of children with T1D for insulin injection. The checklists formally present the order of the technique and all the steps for insulin injection and allow a quantitative assessment of the operational skills of children. The developed instruments offer providers the possibility of continuous assessment of the progress of the pediatric clientele until they reach independence in diabetes self-care.

Checklists to reduce diagnostic error: a systematic review of the literature using a human factors framework

OBJECTIVES

To apply a human factors framework to understand whether checklists reduce clinical diagnostic error have (1) gaps in composition; and (2) components that may be more likely to reduce errors.

DESIGN

Systematic review.

DATA SOURCES

PubMed, EMBASE, Scopus and Web of Science were searched through 15 February 2022.

ELIGIBILITY CRITERIA

Any article that included a clinical checklist aimed at improving the diagnostic process. Checklists were defined as any structured guide intended to elicit additional thinking regarding diagnosis.

DATA EXTRACTION AND SYNTHESIS

Two authors independently reviewed and selected articles based on eligibility criteria. Each extracted unique checklist was independently characterised according to the well-established human factors framework: Systems Engineering Initiative for Patient Safety 2.0 (SEIPS 2.0). If reported, checklist efficacy in reducing diagnostic error (eg, diagnostic accuracy, number of errors or any patient-related outcomes) was outlined. Risk of study bias was independently evaluated using standardised quality assessment tools in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

RESULTS

A total of 30 articles containing 25 unique checklists were included. Checklists were characterised within the SEIPS 2.0 framework as follows: Work Systems subcomponents of Tasks (n=13), Persons (n=2) and Internal Environment (n=3); Processes subcomponents of Cognitive (n=20) and Social and Behavioural (n=2); and Outcomes subcomponents of Professional (n=2). Other subcomponents, such as External Environment or Patient outcomes, were not addressed. Fourteen checklists examined effect on diagnostic outcomes: seven demonstrated improvement, six were without improvement and one demonstrated mixed results. Importantly, Tasks-oriented studies more often demonstrated error reduction (n=5/7) than those addressing the Cognitive process (n=4/10).

CONCLUSIONS

Most diagnostic checklists incorporated few human factors components. Checklists addressing the SEIPS 2.0 Tasks subcomponent were more often associated with a reduction in diagnostic errors. Studies examining less explored subcomponents and emphasis on Tasks, rather than the Cognitive subcomponents, may be warranted to prevent diagnostic errors.

Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam

BACKGROUND

Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine.

METHODS

All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded.

RESULTS

The data from these examiners (n=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, n=60, 2015: 89.8%, n=230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists.

CONCLUSIONS

Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.