Measuring nursing competencies in the operating theatre: instrument development and psychometric analysis using Item Response Theory

Item-Level Analysis of a Newly Developed Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) Using the Rasch Measurement Model

The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a tool designed specifically to observe and measure registered dietitian nutritionists’ (RDNs) nutrition-focused physical exam (NFPE) competence in authentic acute care settings. The initial INSPECT items were generated and tested for content and face validity using expert RDNs’ input. The INSPECT was further examined for inter-rater, intra-rater, and internal consistency using clinical supervisor observations of RDNs performing NFPE on patients in real-life acute care settings. These previous studies showed the INSPECT to have excellent content validity, acceptable face validity, good inter-rater reliability, moderate to strong intra-rater reliability, and excellent internal consistency. In the current study, the Rasch measurement model was applied to examine the item-level properties of the INSPECT. Results confirm that the INSPECT measured a single construct. All items fit the established criteria for clinical observations of >0.5 and <1.7, had positive point measure correlations, met the Wright Unidimensionality Index criteria of ≥0.9, exhibited one latent construct with >40% variance explained by the Rasch dimension as well as a sub-dimension based on item difficulty from the principal component analysis of the first contrast Rasch residuals. Rasch rating scale analysis revealed that the rating scale and majority of the items (39/41) fit the Rasch model. Rasch item hierarchy analysis matched the a priori hypothesized hierarchy for the top-most and bottom-most items. Ceiling effects were seen for three items (hand hygiene, personal protective equipment, and patient position) and one item (handgrip using hand dynamometer) reached the floor effect. Rasch reliability assessment demonstrated high person reliability (0.86), high item reliability (0.96), and person separation of 3.56 ability levels. The principal component analysis of residuals revealed two factors based on item difficulty, one for micronutrient exam and another for macronutrient exam, initial steps, and bedside manner. The resulting two factors may likely be due to a sub-dimension of the latent NFPE trait. Overall, the INSPECT items were found to have good item-level psychometrics. Continued testing of the INSPECT with RDNs at different ability levels will help to determine cut-off scores ranging from novice to expert. Establishing cut-off scores for the INSPECT will further enhance the utility of the tool.

Inter-Rater and Intra-Rater Reliability of the INSPECT (Interactive Nutrition Specific Physical Exam Competency Tool) Measured in Multi-Site Acute Care Settings

Even as new medical modalities, diagnostics, and technologies are rapidly changing healthcare, providing patients with safe, high-quality care remains the central focus. To provide safe patient care, healthcare providers are obligated to demonstrate and maintain the necessary competence. As more healthcare disciplines move toward a competency-based education model, it is essential to extend the competence verification from the academic educational level to the patient’s bedside. The nutrition-focused physical exam (NFPE) is a competency recently adopted by registered dietitian nutritionists (RDNs) for assessing patients’ nutritional status. Being a newly acquired skill, validated tools are required to measure NFPE competence during routine clinical practice. The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a new tool developed specifically to observe and measure RDNs’ NFPE competence in clinical settings. The INSPECT was designed and validated for content using expert RDNs’ input in the first and second phases of the study. This current study aimed to assess the reliability of the INSPECT through multi-site observations by clinical supervisors evaluating RDNs’ NFPE competency during patient assessment. The INSPECT exhibited good inter-rater reliability (ICC = 0.78 for the first assessment and ICC = 0.68 for the second assessment), moderate to strong intra-rater reliability for 37 of 41 items (Spearman rho = 0.54 to 1.0), and excellent internal consistency (Cronbach’s α = 0.86 for the first assessment and α = 0.92 for the second assessment). In total, 10 out of the 11 INSPECT subsets showed good to excellent internal consistency (α ranging from 0.70 to 0.98). The results demonstrate that the INSPECT is a reliable tool, is stable over time, and has good agreement and excellent consistency between raters. The INSPECT can be a valuable tool to measure, promote and maintain RDNs’ NFPE competence in authentic acute care settings.

Assessment of Dysfunctional Grief due to Death from COVID-19 in Peru: Adaptation and Validation of a Spanish Version of the Pandemic Grief Scale

This study evaluated the psychometric properties of the Pandemic Grief Scale (PGS), using Classical Test Theory (CTT) and Item Response Theory (IRT) methods, in a sample of 458 people who have suffered the loss of a loved one due to the COVID-19 pandemic. The Pandemic Grief Scale, Patient Health Questionnaire-2 (PHQ-2), the two-item Generalized Anxiety Disorder Scale (GAD-2), and a single item on suicidal ideation were used. The unidimensional model had good fit and reliability; furthermore, convergent validity was demonstrated based on the relationships between dysfunctional grief, anxiety symptoms, depression, and suicidal ideation. Additionally, a higher presence of dysfunctional grief is required to answer using the higher response categories. The evidence of validity and reliability of the PGS in its Spanish version, using traditional and modern methods, is confirmed in Peru.

Psychometric Testing of the Chinese Simple Version of the Simulation Learning Effectiveness Inventory: Classical Theory Test and Item Response Theory

BACKGROUND

High-fidelity simulation (HFS) has become a widely used and established pedagogy for teaching clinical nursing skills. Nevertheless, there are few evidence-based instruments that validate the effectiveness of simulation learning in mainland China.

METHODS

The Simulation Learning Effectiveness Inventory (SLEI) was adapted and validated for use in this study. Psychometric evaluation, incorporating classical test theory and item response theory (IRT) methods, was performed with 533 third-year undergraduate nursing students who were recruited from May 2017 to July 2018.

RESULTS

The findings of exploratory and confirmatory factor analyses revealed that the simplified Chinese version of the SLEI (SLEI-SC) was composed of six factors, namely, course arrangement, equipment resource, debriefing, clinical ability, problem solving, and confidence, which explained 60.84% of the total variance. The Cronbach's α, MIIC, marginal reliability, and test-retest reliability values obtained for the total scale were 0.88, 0.38, 0.96, and 0.88, respectively. Furthermore, the difference between the total scores for learning effectiveness pre- and post-course was statistically significant (t = 2.59, p < 0.05, Cohen's d = 0.60). IRT analysis showed that the SLEI-SC discriminates well between students with high and low levels of learning effectiveness and offers information about a broad range of learning effectiveness measures. The relationship between final course grade and total score on the SLEI-SC was statistically significant (r = 0.63, p < 0.05).

CONCLUSION

We demonstrated initial psychometric evidence and support for the 31-item SLEI-SC as a developmentally appropriate instrument for assessing the learning effectiveness of all phases of HFS use with nursing students.

Development and Psychometric Evaluation of an Instrument to Measure the Use of Intuition in Clinical Practice by Critical Care Nurses

BACKGROUND

The current study was conducted to develop and validate an instrument to measure the use of intuition in clinical practice by critical care nurses.

METHODS

Item generation and psychometric evaluation were developed. In the psychometric, content validity index and content validity ratio were calculated to establish initial instrument validity through the use of expert ratings, as well as, construct and criterion validity.

RESULTS

The original items reduced to 25. Using principal components analysis and orthogonal varimax rotation, three factors had an eigenvalue >1, with 60.05% variance (Factor 1: 47.9%; Factor 2: 7.56%; and Factor 3: 5.05%). The tool had an acceptable correlation to criterion of the instrument (r = .769, p < .001), a Cronbach alpha consistency of 0.953, and a stability level of r = .945 and p < .001.

CONCLUSION

In this study, a valid and reliable instrument was developed to measure intuition.

Students' approaches to learning in a clinical practicum: A psychometric evaluation based on item response theory

BACKGROUND

The investigation of learning approaches in the clinical workplace context has remained an under-researched area. Despite the validation of learning approach instruments and their applications in various clinical contexts, little is known about the extent to which an individual item, that reflects a specific learning strategy and motive, effectively contributes to characterizing students' learning approaches.

OBJECTIVES

This study aimed to measure nursing students' approaches to learning in a clinical practicum using the Approaches to Learning at Work Questionnaire (ALWQ).

DESIGN

Survey research design was used in the study.

SETTINGS AND PARTICIPANTS

A sample of year 3 nursing students (n = 208) who undertook a 6-week clinical practicum course participated in the study.

METHODS

Factor analyses were conducted, followed by an item response theory analysis, including model assumption evaluation (unidimensionality and local independence), item calibration and goodness-of-fit assessment.

RESULTS

Two subscales, deep and surface, were derived. Findings suggested that: (a) items measuring the deep motive from intrinsic interest and deep strategies of relating new ideas to similar situations, and that of concept mapping served as the strongest discriminating indicators; (b) the surface strategy of memorizing facts and details without an overall picture exhibited the highest discriminating power among all surface items; and, (c) both subscales appeared to be informative in assessing a broad range of the corresponding latent trait. The 21-item ALWQ derived from this study presented an efficient, internally consistent and precise measure.

CONCLUSIONS

Findings provided a useful psychometric evaluation of the ALWQ in the clinical practicum context, added evidence to the utility of the ALWQ for nursing education practice and research, and echoed the discussions from previous studies on the role of the contextual factors in influencing student choices of different learning strategies. They provided insights for clinical educators to measure nursing students' approaches to learning and facilitate their learning in the clinical practicum setting.

Assessment of bachelor's theses in a nursing degree with a rubrics system: Development and validation study

BACKGROUND

Writing a bachelor thesis (BT) is the last step to obtain a nursing degree. In order to perform an effective assessment of a nursing BT, certain reliable and valid tools are required.

OBJECTIVES

To develop and validate a 3-rubric system (drafting process, dissertation, and viva) to assess final year nursing students' BT.

DESIGN

A multi-disciplinary study of content validity and psychometric properties. The study was carried out between December 2014 and July 2015.

SETTINGS

Nursing Degree at Universitat Jaume I. Spain.

PARTICIPANTS

Eleven experts (9 nursing professors and 2 education professors from 6 different universities) took part in the development and content validity stages. Fifty-two theses presented during the 2014-2015 academic year were included by consecutive sampling of cases in order to study the psychometric properties.

METHODS

First, a group of experts was created to validate the content of the assessment system based on three rubrics (drafting process, dissertation, and viva). Subsequently, a reliability and validity study of the rubrics was carried out on the 52 theses presented during the 2014-2015 academic year.

RESULTS

The BT drafting process rubric has 8 criteria (S-CVI=0.93; α=0.837; ICC=0.614), the dissertation rubric has 7 criteria (S-CVI=0.9; α=0.893; ICC=0.74), and the viva rubric has 4 criteria (S-CVI=0.86; α=8.16; ICC=0.895).

CONCLUSION

A nursing BT assessment system based on three rubrics (drafting process, dissertation, and viva) has been validated. This system may be transferred to other nursing degrees or degrees from other academic areas. It is necessary to continue with the validation process taking into account factors that may affect the results obtained.