Bayesian Multilevel MIMIC Modeling for Studying Measurement Invariance in Cross-group Comparisons

Unfinished nursing care in intensive care units and the mediating role of the association between nurse working environment, and quality of care and nurses' wellbeing

OBJECTIVES

Unfinished care refers to the situation in which nurses are forced to delay or omit necessary nursing care. The objectives was: 1) to measure the prevalence of unfinished nursing care in intensive care units during the COVID-19 pandemic; 2) to examine whether unfinished nursing care has a mediating role in the relationship between nurse working environment and nurse-perceived quality of care and risk of burnout among nurses.

DESIGN

A national cross-sectional survey.

SETTING

Seventy-five intensive care units in Belgium (December 2021 to February 2022).

MAIN OUTCOME MEASURES

The Practice Environment Scale of the Nursing Work Index was used to measure the work environment. The perception of quality and safety of care was evaluated via a Likert-type scale. The risk of burnout was assessed using the Maslach Burnout Inventory scale.

RESULTS

A total of 2,183 nurse responses were included (response rate of 47.8%). Seventy-six percent of nurses reported at least one unfinished nursing care activity during their last shift. The staffing and resource adequacy subdimension of the Practice Environment Scale of the Nursing Work Index had the strongest correlation with unfinished nursing care. An increase in unfinished nursing care led to significantly lower perceived quality and safety of care and an increase in high risk of burnout. Unfinished nursing care appears to be a mediating factor for the association between staffing and resource adequacy and the quality and safety of care perceived by nurses and risk of burnout.

CONCLUSIONS

Unfinished nursing care, which is highly related to staffing and resource adequacy, is associated with increased odds of nurses being at risk of burnout and reporting a lower level of perceived quality of care.

IMPLICATIONS FOR CLINICAL PRACTICE

The monitoring of unfinished nursing care in the intensive care unit is an important early indicator of problems related to adequate staffing levels, the well-being of nurses, and the perceived quality of care.

Invariance of the WHO violence against women instrument among Kenyan adolescent girls and young women: Bayesian psychometric modeling

INTRODUCTION

To make valid comparisons across groups, a measurement instrument needs to be measurement invariant across those groups. The present study evaluates measurement invariance for experience of violence among adolescent girls and young women (AGYW) in two informal settlements in Nairobi, Kenya.

METHODS

We used survey data collected from 1,081 AGYW aged 15-22 years from two Nairobi's informal settlements of Korogocho (n = 617) and Viwandani (n = 464) in 2017 through DREAMS (an initiative aimed at reducing HIV incidence among AGYW with a core package of evidence-based interventions) impact evaluation project. Experience of violence was measured using the 15-item WHO's violence against women instrument, and factorial (non)invariance assessed within exploratory structural equation modeling (ESEM) framework. Cross-group measurement invariance was assessed using Bayesian Multiple Indicator Multiple Causes (MIMIC) model across site, age groups, self-reported invitation to participate in DREAMS, marital status, currently in school, education level, religion, ethnic groups, ever had sex, slept hungry at night past 4 weeks, and wealth index.

RESULTS

The mean and median ages of the AGYW were 17.9 years and 17 years, respectively. About 59% reported having had sex and 58% of AGYW were in school. The percentage reporting each act of violence varied from 1.6% ("attacked you with a weapon") to 26.5% ("insult you or make you feel bad about yourself"). About 44% (n = 474) of participants experienced ≥1 acts of violence, and 2.7% (n = 29) experienced at least half of the 15 acts. The structure underlying the 15 items was configurally similar to that proposed by WHO, with three factors reflecting either psychological, physical, or sexual violence. Noninvariance was detected for five items-spread across the three domains. Three of five items showed noninvariance only for sleeping hungry at night in the past 4 weeks. As the majority of items did not show evidence of noninvariance, differences in latent mean scores likely reflect actual differences and may not be attributable to measurement artifacts.

CONCLUSIONS

Using state-of-the-art statistical techniques on a widely used instrument for measuring exposure to violence among women, this study provides support for the subscales of psychological, physical and sexual violence in a Kenyan AGYW population. The instrument supports comparisons across groups within this population. This is crucial when comparing violence against girls/women prevalence rates and to understand challenges and exchange strategies to reduce abuse or violence experienced by AGYW, or women in general.

Validity Evidence Based on Relations to Other Variables of the eHealth Literacy Questionnaire (eHLQ): Bayesian Approach to Test for Known-Groups Validity

Background

As health resources and services are increasingly delivered through digital platforms, eHealth literacy is becoming a set of essential capabilities to improve consumer health in the digital era. To understand eHealth literacy needs, a meaningful measure is required. Strong initial evidence for the reliability and construct validity of inferences drawn from the eHealth Literacy Questionnaire (eHLQ) was obtained during its development in Denmark, but validity testing for varying purposes is an ongoing and cumulative process.

Objective

This study aims to examine validity evidence based on relations to other variables—using data collected with the known-groups approach—to further explore if the eHLQ is a robust tool to understand eHealth literacy needs in different contexts. A priori hypotheses are set for the expected score differences among age, sex, education, and information and communication technology (ICT) use for each of the 7 eHealth literacy constructs represented by the 7 eHLQ scales.

Methods

A Bayesian mediated multiple indicators multiple causes model approach was used to simultaneously identify group differences and test measurement invariance through differential item functioning across the groups, with ICT use as a mediator. A sample size of 500 participants was estimated. Data were collected at 3 diverse health sites in Australia.

Results

Responses from 525 participants were included for analysis. Being older was significantly related to lower scores in 4 eHLQ scales, with 3. Ability to actively engage with digital services having the strongest effect (total effect –0.37; P<.001), followed by 1. Using technology to process health information (total effect –0.32; P<.001), 5. Motivated to engage with digital services (total effect –0.21; P=.01), and 7. Digital services that suit individual needs (total effect –0.21; P=.02). However, the effects were only partially mediated by ICT use. Higher education was associated with higher scores in 1. Using technology to process health information (total effect 0.22; P=.01) and 3. Ability to actively engage with digital services (total effect 0.25; P<.001), with the effects mostly mediated by ICT use. Higher ICT use was related to higher scores in all scales except 2. Understanding health concepts and language and 4. Feel safe and in control. Either no or ignorable cases of differential item functioning were found across the 4 groups.

Conclusions

By using a Bayesian mediated multiple indicators multiple causes model, this study provides supportive validity evidence for the eHLQ based on relations to other variables as well as established evidence regarding internal structure related to measurement invariance across the groups for the 7 scales in the Australian community health context. This study also demonstrates that the eHLQ can be used to gain valuable insights into people’s eHealth literacy needs to help optimize access and use of digital health and promote health equity.

Validation of the Psychometric Properties of the Practice Environment Scale of Nursing Work Index in Primary Health Care in Portugal

Studies related to the work environment in primary health care are scarce in the literature. The present study aimed to validate the psychometric properties of the Practice Environment Scale of Nursing Work Index (PES-NWI) in primary health care (PHC) and to evaluate its construct validity through exploratory and confirmatory factor analyses in a sample of Portuguese nurses. A quantitative, cross-sectional, and validation study design was implemented. Methods: The sample consisted of 1059 nurses from the PHC units of all 55 health center groups (HCGs) in mainland Portugal, 15 health centers in the Autonomous Region of Madeira, and 6 health centers in the Autonomous Region of the Azores. The study tested different structural models using exploratory and confirmatory factor analysis techniques. The reliability of the scale was tested by determining Cronbach’s alpha coefficient. Results: The internal consistency of the PES-NWI was 0.91. Exploratory and confirmatory factor analyses were performed on the PES-NWI model in PHC with five factors: NPOA, NFQC, NMALSN, SRA, and CNPR. The results show that the scale presents acceptable fit quality indexes in the final factorial solution and adequate convergent validity. Conclusion: The PES-NWI in PHC has an adequate, robust, and reliable five-factor structure. The scale is valid and can be used in clinical practice, nursing management, and PHC research.

Examining the measurement invariance of the Practice Environment Scale-Nursing Work Index in the United States

The increasingly diverse nursing workforce in the United States often creates challenges for researchers in meaningfully interpreting survey data from large samples of nurses with heterogeneous cultural backgrounds. Considering this, the purpose of this study was to determine the measurement invariance of the Practice Environment Scale-Nursing Work Index (PES-NWI) between registered nurses (RNs) reporting as Asian/Pacific Islander and those reporting as White. Data from the National Database of Nursing Quality Indicators®, collected in 2018 from RNs (N = 27,619) providing direct patient care, were used in this study. Multi-group confirmatory factor analysis was used to determine measurement invariance. Interpretation of the fit indices across models (configural, metric, and scalar) provides support for measurement invariance of the PES-NWI across the two groups. There is empirical support for pooling PES-NWI data and comparing it across these two groups.

Does the Partners in Health scale allow meaningful comparisons of chronic condition self-management between men and women? Testing measurement invariance

AIMS

To determine if the Partners in Health scale, pertinent to assessing patient chronic condition self-management, operates equivalently for men and women.

BACKGROUND

There are distinct gender-based differences in self-management behaviours and health perceptions. This may introduce non-invariance in self-report measures. Testing of measurement invariance is a recommended practice in nursing science to ensure robust metrics.

DESIGN

A representative cross-sectional population survey in South Australian.

METHOD

In 2014, 940 people responded to the South Australian Health Omnibus Survey, a battery of health-related questions. MI and estimation of heterogeneity was tested using Bayesian confirmatory factor analysis.

RESULTS

Findings showed self-management constructs were interpreted equivalently between men and women. Observed population heterogeneity associated lower education levels with poorer illness and treatment knowledge, smokers with poorer treatment partnerships and mental health problems with lower coping capacity.

CONCLUSION

Approximate measurement invariance was achieved between men and women for Partners in Health scale.

IMPACT

There is a lack of well-validated generic instruments, including investigation into gender variability, for measuring chronic condition self-management behaviours. Lower education levels were found to connect with poorer knowledge of health condition and treatment. Mental health problems attenuated ability to cope with the effect of the condition. Findings can facilitate the development of better tailored interventions for self-management of patients' chronic condition/s.

Assessing Online Flow Across Cultures: A Two-Fold Measurement Invariance Study

The association between online Flow and Internet Gaming Disorder (IGD) has attracted significant attention. Despite the consensus that online Flow plays a pivotal role in the development of IGD and other Internet addictive behaviors, there has been a lack of consistency in measurement scales used to assess online Flow. Even widely used measures of online Flow have not been psychometrically assessed across culturally diverse populations of gamers. Such an assessment would enhance the accuracy of cross-cultural comparisons. Attending to this need, the present study assessed the psychometric properties of the binary coded (i.e., Yes, No) Online Flow Questionnaire (OFQ), while concurrently taking into consideration country, age, language, and mode of data collection (online or face-to-face) differences. Two sequences of successive multiple group confirmatory factor analyses were used to assess the psychometric properties of the OFQ, between: (a) emergent adults from the United States of America (N = 482, Mage = 25.23, SD = 2.746) and Australia (N = 168, Mage = 23.55, SD = 3.37) and (b) adolescents from Greece (N = 1579, Mage = 16.12, SD = 0.849) and Cyprus (N = 1372, Mage = 15.54, SD = 0.656). Configural and partial metric invariance were confirmed between the United States and Australian samples. For the Greek and Cypriot samples, results indicated full configural and metric invariance. These results provide initial information to researchers and clinicians of the extent to which the OFQ maintains its consistency when used across cultures and invite for further cross-cultural studies in the field. Implications, as well as limitations, are discussed.

Practice environment scale of the nursing work index: Portuguese version and psychometric properties

AIM

To assess the psychometric properties regarding the Portuguese version of the Practice Environment Scale of the Nursing Work Index.

BACKGROUND

The Practice Environment Scale of the Nursing Work Index is the most widely used measure for assessing the practice environment. A model with a higher number of factors appears to be more adequate to Portuguese settings.

METHOD

A confirmatory factor analysis was performed to the Practice Environment Scale of the Nursing Work Index using a sample of 850 Portuguese nurses. Three models were tested: the original model, a seven-factor model and a higher-order model. An analysis of invariance was performed in two subsets to confirm the stability of the solution.

RESULTS

The seven-factor model fit better to the data than the original model. After refinement, this solution showed suitability and a stable factor structure. Reliability, convergent validity and discriminant validity were confirmed. A second-order factor solution also showed suitability.

CONCLUSION

The seven-factor structure of the Practice Environment Scale of the Nursing Work Index showed a better goodness-of-fit to Portuguese settings than the original structure. The second-order factor solution allows an overall assessment of practice environments.

IMPLICATIONS FOR NURSING MANAGEMENT

The Portuguese version of the Practice Environment Scale of the Nursing Work Index is a valuable tool for assessing Portuguese nursing practice environments. The seven-factor solution of the Practice Environment Scale of the Nursing Work Index showed high specificity.

Impact of Cross-level Measurement Noninvariance on Hospital Rankings Based on Patient Experiences With Care in 7 European Countries

BACKGROUND

Hospital-level findings on patient experiences with care are increasingly reported publicly. A critical aspect left unexamined is the commonality of composite measures of patient experiences across different groups of patients, nursing units, hospitals, and countries. Absence of commonality is termed measurement noninvariance and is hypothesized to have a strong impact on performance assessment.

AIM

The aim of this study is to examine measurement invariance across groups and levels under study (patients, nursing units, hospitals, and countries) and illustrate the degree to which this method of analysis impacts hospital rankings.

RESEARCH DESIGN

Data were collected from 11,289 patients in 7 European countries, 186 hospitals, and 824 nursing units. Multilevel factor analytic models were applied to evaluate measurement invariance across the hierarchical levels of the study and across groups at specific levels (self-perceived health at patient level; unit speciality at nursing unit level). Hospital rankings for the final multilevel model were compared with those from a single-level factor model that is unsuspecting of measurement invariance.

RESULTS

Cross-group invariance was shown for levels of self-perceived health and to a large degree also for nursing unit speciality. Patient experience composite measures were, however, not invariant across patient, unit, and hospital levels. Hospital rankings were largely impacted when accounted for this cross-level invariance. The percentage of hospitals with discordant ranks by >10 percentile points varied from 26.7% in Spain to 70% in Poland.

CONCLUSIONS

Leaving unexamined possible noninvariance across groups and hierarchical levels may have far reaching consequences for how the public perceives hospitals' position relative to other hospitals.

Modeling agreement on bounded scales

Agreement is an important concept in medical and behavioral sciences, in particular in clinical decision making where disagreements possibly imply a different patient management. The concordance correlation coefficient is an appropriate measure to quantify agreement between two scorers on a quantitative scale. However, this measure is based on the first two moments, which could poorly summarize the shape of the score distribution on bounded scales. Bounded outcome scores are common in medical and behavioral sciences. Typical examples are scores obtained on visual analog scales and scores derived as the number of positive items on a questionnaire. These kinds of scores often show a non-standard distribution, like a J- or U-shape, questioning the usefulness of the concordance correlation coefficient as agreement measure. The logit-normal distribution has shown to be successful in modeling bounded outcome scores of two types: (1) when the bounded score is a coarsened version of a latent score with a logit-normal distribution on the [0,1] interval and (2) when the bounded score is a proportion with the true probability having a logit-normal distribution. In the present work, a model-based approach, based on a bivariate generalization of the logit-normal distribution, is developed in a Bayesian framework to assess the agreement on bounded scales. This method permits to directly study the impact of predictors on the concordance correlation coefficient and can be simply implemented in standard Bayesian softwares, like JAGS and WinBUGS. The performances of the new method are compared to the classical approach using simulations. Finally, the methodology is used in two different medical domains: cardiology and rheumatology.

Measuring chronic condition self-management in an Australian community: factor structure of the revised Partners in Health (PIH) scale

PURPOSE

To evaluate the factor structure of the revised Partners in Health (PIH) scale for measuring chronic condition self-management in a representative sample from the Australian community.

METHODS

A series of consultations between clinical groups underpinned the revision of the PIH. The factors in the revised instrument were proposed to be: knowledge of illness and treatment, patient-health professional partnership, recognition and management of symptoms and coping with chronic illness. Participants (N = 904) reporting having a chronic illness completed the revised 12-item scale. Two a priori models, the 4-factor and bi-factor models were then evaluated using Bayesian confirmatory factor analysis (BCFA). Final model selection was established on model complexity, posterior predictive p values and deviance information criterion.

RESULTS

Both 4-factor and bi-factor BCFA models with small informative priors for cross-loadings provided an acceptable fit with the data. The 4-factor model was shown to provide a better and more parsimonious fit with the observed data in terms of substantive theory. McDonald's omega coefficients indicated that the reliability of subscale raw scores was mostly in the acceptable range.

CONCLUSION

The findings showed that the PIH scale is a relevant and structurally valid instrument for measuring chronic condition self-management in an Australian community. The PIH scale may help health professionals to introduce the concept of self-management to their patients and provide assessment of areas of self-management. A limitation is the narrow range of validated PIH measurement properties to date. Further research is needed to evaluate other important properties such as test-retest reliability, responsiveness over time and content validity.

Experienced pediatric nurses' perceptions of work-related stressors on general medical and surgical units: A qualitative study

BACKGROUND

Experienced pediatric nurses caring for increasingly sick and vulnerable children on medical and surgical units may be at particular risk for work-related stress. In view of their positive impact on quality of care, and the fact that they are particularly difficult to retain, it is imperative to understand the work-related stressors these nurses encounter in order to develop effective organizational interventions to minimize stressors and promote retention.

OBJECTIVE

To explore experienced pediatric nurses' perceptions of work-related stressors in medical and surgical units.

DESIGN

Qualitative descriptive design with semi-structured interviews.

SETTING

Medical and surgical units at a quaternary care pediatric hospital in Montreal, Canada.

PARTICIPANTS

Nurses recognized as experienced by the nursing leadership team as reflected by having been 'in charge' of the unit, or having trained junior staff, and who had been practicing full-time for three years or more on a general medical or surgical pediatric unit were eligible to participate. Purposive sampling was used, and nurses recruited until data saturation was reached (n=12). There were no refusals to participate.

METHODS

Semi-structured interviews were conducted between August and December 2013.

RESULTS

Nurses described a strong sense of responsibility for providing excellent patient care, and identified stressor that negatively impacted their ability to do so. Stressors are reflected in three themes: (1) "The kids are getting sicker and sicker": Difficulty ensuring excellent patient care to an increasingly vulnerable population, (2) Feeling powerless to provide quality care, and (3) Being a "Jack-of-all-trades": Struggling with competing demands.

CONCLUSION

Experienced pediatric nurses felt powerless to provide quality care to an increasingly acute and vulnerable population. Dealing with multiple and diverse responsibilities, and limited resources and support, were important stressors. Nurse Managers and educators could mitigate stressors and improve retention of experienced pediatric nurses by offering targeted continuing education to those newly responsible for additional roles, and building supportive working environments that encourage collaboration and empower experienced nurses.

Organization of Hospital Nursing, Provision of Nursing Care, and Patient Experiences With Care in Europe

This study integrates previously isolated findings of nursing outcomes research into an explanatory framework in which care left undone and nurse education levels are of key importance. A moderated mediation analysis of survey data from 11,549 patients and 10,733 nurses in 217 hospitals in eight European countries shows that patient care experience is better in hospitals with better nurse staffing and a more favorable work environment in which less clinical care is left undone. Clinical care left undone is a mediator in this relationship. Clinical care is left undone less frequently in hospitals with better nurse staffing and more favorable nurse work environments, and in which nurses work less overtime and are more experienced. Higher proportions of nurses with a bachelor's degree reduce the effect of worse nurse staffing on more clinical care left undone.