Reliability estimation in a multilevel confirmatory factor analysis framework

Covariate-free and Covariate-dependent Reliability

Classical test theory reliability coefficients are said to be population specific. Reliability generalization, a meta-analysis method, is the main procedure for evaluating the stability of reliability coefficients across populations. A new approach is developed to evaluate the degree of invariance of reliability coefficients to population characteristics. Factor or common variance of a reliability measure is partitioned into parts that are, and are not, influenced by control variables, resulting in a partition of reliability into a covariate-dependent and a covariate-free part. The approach can be implemented in a single sample and can be applied to a variety of reliability coefficients.

Multilevel Factor Analysis: Reporting Guidelines and a Review of Reporting Practices

We provide reporting guidelines for multilevel factor analysis (MFA) and use these guidelines to systematically review 72 MFA applications in journals across a range of disciplines (e.g., education, health/nursing, management, and psychology) published between 1994 and 2014. Results are organized in terms of the (a) characteristics of the MFA application (e.g., construct measured), (b) purpose (e.g., measurement validation), (c) data source (e.g., number of cases at Level 1 and Level 2), (d) statistical approach (e.g., maximum likelihood), and (e) results reported (e.g., intraclass correlations for indicators and latent variables, standardized factor loadings, fit indices). Results from this review have implications for applied researchers interested in expanding their approaches to psychometric analyses and construct validation within a multilevel framework and for methodologists using Monte Carlo methods to explore technical and methodological issues grounded in realistic research design conditions.

Reappraisal Buffers the Association between Stress and Negative Mood Measured over 14 Days: Implications for Understanding Psychological Resilience

Reappraisal is thought to be an adaptive emotion regulation strategy, and research suggests that individuals who habitually reappraise report more positive patterns of affect overall. However, some experimental studies indicate that a greater tendency to reappraise can exacerbate stress response, and it is unclear whether reappraisal confers resilience or exacerbates response to naturally occurring stressors. In order to address this, the present study investigated whether reappraisal prospectively moderated the association between daily stressors and daily negative mood measured over 14 days. Participants (n = 236) completed a measure of reappraisal at baseline, before completing daily online entries of stress and positive and negative mood. Data were analysed using multilevel modelling. Results suggested that reappraisal moderated the association between stress and negative mood, such that higher levels of reappraisal were associated with lower levels of negative mood in response to stress. Moreover, higher reappraisal was also independently associated with lower levels of daily negative mood and higher levels of positive mood. These results suggest that higher reappraisal may confer resilience to stress. Copyright © 2016 European Association of Personality Psychology

The Impact of Non-attempted and Dually-Attempted Items on Person Abilities Using Item Response Theory

The purpose of the present study was to relate response strategy with person ability estimates. Two behavioral strategies were examined: (a) the strategy to skip items in order to save time on timed tests, and, (b) the strategy to select two responses on an item, with the hope that one of them may be considered correct. Participants were 4,422 individuals who were administered a standardized achievement measure related to math, biology, chemistry, and physics. In the present evaluation, only the physics subscale was employed. Two analyses were conducted: (a) a person-based one to identify differences between groups and potential correlates of those differences, and, (b) a measure-based analysis in order to identify the parts of the measure that were responsible for potential group differentiation. For (a) person abilities the 2-PL model was employed and later the 3-PL and 4-PL models in order to estimate upper and lower asymptotes of person abilities. For (b) differential item functioning, differential test functioning, and differential distractor functioning were investigated. Results indicated that there were significant differences between groups with completers having the highest ability compared to both non-attempters and dual responders. There were no significant differences between no-attempters and dual responders. The present findings have implications for response strategy efficacy and measure evaluation, revision, and construction.

Examining the relationship between home literacy environment and neural correlates of phonological processing in beginning readers with and without a familial risk for dyslexia: an fMRI study

Developmental dyslexia is a language-based learning disability characterized by persistent difficulty in learning to read. While an understanding of genetic contributions is emerging, the ways the environment affects brain functioning in children with developmental dyslexia are poorly understood. A relationship between the home literacy environment (HLE) and neural correlates of reading has been identified in typically developing children, yet it remains unclear whether similar effects are observable in children with a genetic predisposition for dyslexia. Understanding environmental contributions is important given that we do not understand why some genetically at-risk children do not develop dyslexia. Here, we investigate for the first time the relationship between HLE and the neural correlates of phonological processing in beginning readers with (FHD+, n = 29) and without (FHD-, n = 21) a family history of developmental dyslexia. We further controlled for socioeconomic status to isolate the neurobiological mechanism by which HLE affects reading development. Group differences revealed stronger correlation of HLE with brain activation in the left inferior/middle frontal and right fusiform gyri in FHD- compared to FHD+ children, suggesting greater impact of HLE on manipulation of phonological codes and recruitment of orthographic representations in typically developing children. In contrast, activation in the right precentral gyrus showed a significantly stronger correlation with HLE in FHD+ compared to FHD- children, suggesting emerging compensatory networks in genetically at-risk children. Overall, our results suggest that genetic predisposition for dyslexia alters contributions of HLE to early reading skills before formal reading instruction, which has important implications for educational practice and intervention models.

Measurement Error Correction Formula for Cluster-Level Group Differences in Cluster Randomized and Observational Studies

Multilevel modeling (MLM) is frequently used to detect cluster-level group differences in cluster randomized trial and observational studies. Group differences on the outcomes (posttest scores) are detected by controlling for the covariate (pretest scores) as a proxy variable for unobserved factors that predict future attributes. The pretest and posttest scores that are most often used in MLM are total scores. In prior research, there have been concerns regarding measurement error in the use of total scores in using MLM. In this article, using ordinary least squares and an attenuation formula, we derive the measurement error correction formula for cluster-level group difference estimates from MLM in the presence of measurement error in the outcome, the covariate, or both. Examples are provided to illustrate the correction formula in cluster randomized and observational studies using between-cluster reliability coefficients recently developed.

Multilevel factor analysis of smokers' real-time negative affect ratings while quitting

Smoking is a serious public health problem, and accurate real-time assessment of risk factors associated with smoking is critical to understanding smoking relapse. Negative affect is often described as a critical risk factor related to smoking relapse, and ecological momentary assessment (EMA) methods have been widely used to study real-time relations between negative affect and smoking. However, the factor structure of momentary negative affect ratings is unknown. The current investigation examined the multilevel factor structure and internal consistency of an EMA measure of negative affect. Daily assessments were collected for 1 week prequit and 3 weeks postquit from 113 adult daily smokers receiving nicotine replacement therapy and counseling to quit smoking. Results supported a 2-factor model with correlated but distinct agitation and distress factors, rather than a single-factor model of negative affect. The agitation factor was indicated by these items: impatient, tense/anxious, restless. The distress factor was indicated by these items: sad/depressed, upset, distressed. The 2-factor model had acceptable model fit and consistent factor loadings across 3 separate cessation phases: prequit, postquit with recent smoking, and postquit without recent smoking. The 2 factors were highly correlated, showed good internal consistency, and showed strong associations with theoretically relevant smoking and affect variables. Agitation was more strongly related to urge to smoke, and distress was more strongly related to recent stress. This study provides support for a 2-factor model of an EMA measure of negative affect and highlights distinct facets that may be useful for future investigations of affect and smoking. (PsycINFO Database Record

Mediators of the relationship between thin-ideal internalization and body dissatisfaction in the natural environment

Social comparisons (i.e., body, eating, exercise) and body surveillance were tested as mediators of the thin-ideal internalization-body dissatisfaction relationship using ecological momentary assessment (EMA). Participants were 232 college women who completed a 2-week EMA protocol, responding to questions three times per day. Multilevel path analysis was used to examine a 2-1-1 mediation model (thin-ideal internalization assessed as trait; between-person effects examined) and a 1-1-1 model (component of thin-ideal internalization [thin-ideal importance] assessed momentarily; within- and between-person effects examined). For the 2-1-1 model, only body comparison and body surveillance were significant specific mediators of the between-person effect. For the 1-1-1 model, all four variables were significant specific mediators of the within-person effect. Only body comparison was a significant specific mediator of the between-person effect. At the state level, many processes explain the thin-ideal internalization-body dissatisfaction relationship. However, at the trait level, body comparison and body surveillance are more important explanatory factors.

Political skill dimensions and transformational leadership in China

Purpose

– The purpose of this paper is to explore the relationship between the sub-dimensions of political skill and transformational leadership, arguing that in a Chinese sample, social astuteness, networking ability, and interpersonal influence will have a stronger impact than apparent sincerity. Additionally, transformational leadership is argued to mediate the relationship between leader political skill and subordinate job performance.

Design/methodology/approach

– This study used a dyadic, cross-sectional design where all data were collected at a single point in time and supervisors were matched to their followers.

Findings

– The results support that transformational leadership mediates the relationship between leader political skill and follower performance. Additionally, social astuteness was positively related to leader charisma, individualized consideration, and intellectual stimulation; whereas, interpersonal influence was related to leader charisma and intellectual stimulation. Apparent sincerity was not related to perceptions of transformational leadership.

Practical implications

– The findings imply that the authors might be able to design more customized training to reduce the costs of leadership development programs. Specially, in China, focus should be paid to increasing social astuteness and interpersonal influence only.

Originality/value

– This study is one of the few to examine the relationship between the sub-dimensions of political skill and transformational leadership. This is especially noteworthy in the context of the Chinese sample employed, as most political skill research has been done in western contexts.

Within-Day Time-Varying Associations Between Behavioral Cognitions and Physical Activity in Adults

This study used time-varying effect modeling to examine time-of-day differences in how behavioral cognitions predict subsequent physical activity (PA). Adults (N = 116) participated in three 4-day "bursts" of ecological momentary assessment (EMA). Participants were prompted with eight EMA questionnaires per day assessing behavioral cognitions (i.e., intentions, self-efficacy, outcome expectations) and wore an accelerometer during waking hours. Subsequent PA was operationalized as accelerometer-derived minutes of moderate- or vigorousintensity PA in the 2 hr following the EMA prompt. On weekdays, intentions positively predicted subsequent PA in the morning (9:25 a.m.-11:45 a.m.) and in the evening (8:15 p.m.-10:00 p.m.). Self-efficacy positively predicted subsequent PA on weekday evenings (7:35 p.m.-10:00 p.m.). Outcome expectations were unrelated to subsequent PA on weekdays. On weekend days, behavior cognitions and subsequent PA were unrelated regardless of time of day. This study identifies windows of opportunity and vulnerability for motivation-based PA interventions aiming to deliver intervention content within the context of adults' daily lives.

Measuring chronic condition self-management in an Australian community: factor structure of the revised Partners in Health (PIH) scale

PURPOSE

To evaluate the factor structure of the revised Partners in Health (PIH) scale for measuring chronic condition self-management in a representative sample from the Australian community.

METHODS

A series of consultations between clinical groups underpinned the revision of the PIH. The factors in the revised instrument were proposed to be: knowledge of illness and treatment, patient-health professional partnership, recognition and management of symptoms and coping with chronic illness. Participants (N = 904) reporting having a chronic illness completed the revised 12-item scale. Two a priori models, the 4-factor and bi-factor models were then evaluated using Bayesian confirmatory factor analysis (BCFA). Final model selection was established on model complexity, posterior predictive p values and deviance information criterion.

RESULTS

Both 4-factor and bi-factor BCFA models with small informative priors for cross-loadings provided an acceptable fit with the data. The 4-factor model was shown to provide a better and more parsimonious fit with the observed data in terms of substantive theory. McDonald's omega coefficients indicated that the reliability of subscale raw scores was mostly in the acceptable range.

CONCLUSION

The findings showed that the PIH scale is a relevant and structurally valid instrument for measuring chronic condition self-management in an Australian community. The PIH scale may help health professionals to introduce the concept of self-management to their patients and provide assessment of areas of self-management. A limitation is the narrow range of validated PIH measurement properties to date. Further research is needed to evaluate other important properties such as test-retest reliability, responsiveness over time and content validity.

Coefficient α as a Measure of Test Score Reliability: Review of 3 Popular Misconceptions

We discuss 3 popular misconceptions about Cronbach α or coefficient α, traditionally used in public health and the behavioral sciences as an index of test score reliability. We also review several other indices of test score reliability. We encourage researchers to thoughtfully consider the nature of their data and the options when choosing an index of reliability, and to clearly communicate this choice and its implications to their audiences.

Stability and fluctuation of personality disorder features in daily life

Very little is known about the daily stability and fluctuation of personality pathology. To address this gap in knowledge, we investigated the naturalistic manifestation of personality pathology over the course of 100 days. A group of individuals (N = 101) diagnosed with any personality disorder (PD) completed a daily diary study over 100 consecutive days (Mdn = 94 days, range = 33-101 days). Participants completed daily ratings of 30 manifestations of personality pathology. Patterns of stability and variability over the course of the study were then examined. Results indicated that individual PD manifestations and domains of PD manifestations were variable across days and differed widely in their frequency. Additionally, individual averages and level of variability in PD domains were highly stable across months, individual averages of PD domains were predicted by baseline dispositional ratings of PD traits with a high degree of specificity, and daily variability PD domains was associated with elevated levels of PD traits. This pattern of findings suggests that dynamic processes of symptom exacerbation and diminution that are stable in mean level and variability in expression over time characterizes personality pathology. Further, dispositional ratings are significant predictors of average daily expression of PD features. (PsycINFO Database Record

New Reliability and Validity Evidence of the Emotional Intelligence Scale

Emotional Intelligence Scale (EIS) is a popular EI measure. Yet, it has been criticized for an unclear factor structure, and its psychometric properties were mainly examined in the Western context. This study was to evaluate its psychometric properties based on 1,724 Hong Kong undergraduate students, including its (a) factor structure, (b) internal consistency, and (c) criterion validity. We compared different factor structures reported in the literature. The confirmatory factor analysis (CFA) results supported a six-factor structure, which is tallied with Salovey and Mayer’s EI conceptualization. A multigroup CFA also rendered the structure as gender invariant. The scale was internally consistent with high McDonald’s omega coefficients. Significant association between EI and grade point average (GPA) was revealed in the faculties with people-oriented studies. Furthermore, EI was correlated with social, cognitive, and self-growth outcomes and satisfaction of university experience. The study contributes to clarify the factor structure and provides new reliability and validity evidence of the EIS in the Eastern context.

Velocity Explains the Links between Personality States and Affect

The present research examined whether perceived rate of progress toward a goal (velocity) mediated the relationships between personality states and affective states. Drawing from control theories of self-regulation, we hypothesized (i) that increased velocity would mediate the association between state extraversion and state positive affect, and (ii) that decreased velocity would mediate the association between state neuroticism and state negative affect. We tested these hypotheses in 2 experience sampling methodology studies that each spanned 2 weeks. Multilevel modeling analyses showed support for each of the bivariate links in our model, and multilevel path analyses supported our mediation hypotheses. We discuss implications for understanding the relations between personality states and affective states, control theories of self-regulation, and goal striving.

The effects of social anxiety on interpersonal evaluations of warmth and dominance

Social anxiety disorder is associated with interpersonal dysfunction, but it is not clear why people with the disorder feel unsatisfied with their relationships. One possibility is that higher social anxiety could lead to changes in sensitivity to interpersonal traits. We examined whether social anxiety moderates the types of interpersonal evaluations people make regarding warmth and dominance. We developed vignettes in which central characters systematically varied in dominance and warmth and asked two samples of participants (undergraduate students, n=176, and online workers, n=403) to rate their willingness to interact with, and the social desirability of, these characters. Participants in general reported stronger desire to interact with warmer and less dominant characters, and rated warmer and more dominant characters as being more socially desirable. People with higher social anxiety exhibited greater tolerance for colder and more submissive characters on both rated dimensions. The perceived similarity of the characters accounted for the bulk of these effects. Participants indicated a higher desire to interact with characters more similar to themselves, and people with higher social anxiety were more likely to rate submissive and cold characters as being like themselves. The results have implications for clinical interventions for social anxiety disorder.

Ambulatory and diary methods can facilitate the measurement of patient-reported outcomes

PURPOSE

Ambulatory and diary methods of self-reported symptoms and well-being have received increasing interest in recent years. These methods are a valuable addition to traditional strategies for the assessment of patient-reported outcomes (PROs) in that they capture patients' recent symptom experiences repeatedly in their natural environments. In this article, we review ways that incorporating diary methods into PRO measurement can facilitate research on quality of life.

METHODS

Several diary methods are currently available, and they include "real-time" (Ecological Momentary Assessment) and "near-real-time" (end-of-day assessments, Day Reconstruction Method) formats. We identify the key benefits of these methods for PRO research.

RESULTS

(1) In validity testing, diary assessments can serve as a standard for evaluating the ecological validity and for identifying recall biases of PRO instruments with longer-term recall formats. (2) In research and clinical settings, diaries have the ability to closely capture variations and dynamic changes in quality of life that are difficult or not possible to obtain from traditional PRO assessments. (3) In test construction, repeated diary assessments can expand understanding of the measurement characteristics (e.g., reliability, dimensionality) of PROs in that parameters for differences between people can be compared with those for variation within people.

CONCLUSIONS

Diary assessment strategies can enrich the repertoire of PRO assessment tools and enhance the measurement of patients' quality of life.

Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ)

In higher education, student ratings are often used to evaluate and improve the quality of courses and professors’ instructional skills. Unfortunately, student-rating questionnaires rarely generate specific feedback for professors to improve their instructional skills. The impact of student ratings on professors’ instructional skills has proven to be low. This study concerns the psychometric properties of the Instructional Skills Questionnaire (ISQ), a new theory-based student-rating-of-teaching questionnaire with specific questions concerning lecturing skills. The ISQ is administered after a single lecture. This way, it serves as a formative feedback instrument for university professors during courses to assist them to improve and (re-) evaluate their skills if necessary. The ISQ contains seven dimensions of professors’ instructional skills and three student (self perceived) learning outcomes. In this study, Dutch students in 75 courses rated three 90-minute lectures (T1, T2 and T3) of their respective professors using the ISQ. In total, 14,298 ISQ-forms were used to rate 225 lectures. The teacher level reliabilities of the seven dimensions were found to be good at each measurement occasion. In addition, confirmatory multilevel factor analysis confirmed a seven dimensional factor structure at the teacher level at each measurement occasion. Furthermore, specific teacher level factors significantly predicted students’ (self-assessed) learning outcomes. These results partly supported the proposed theoretical framework on the relationship between the ISQ teaching dimensions and the student learning process, and provided evidence for the construct validity of the instrument. In sum, the ISQ is found to be a reliable and valid instrument, which can be used by professors and faculty development centers to assess and improve university teaching.

Perfectionistic Concerns, Social Negativity, and Subjective Well-Being: A Test of the Social Disconnection Model

Partner-specific perfectionistic concerns (PC) include concern over mistakes, self-criticism, and socially prescribed perfectionism as it pertains to one's partner. The social disconnection model proposes that PC influences well-being indirectly through interpersonal problems. Thus, we hypothesized that social negativity (expressed anger, hostility, and rejection) would mediate the relationship between dyadic PC and subjective well-being. Data from 203 romantic dyads (92.1% heterosexual) were collected using self-report surveys and a four-wave, 4-week longitudinal design. Participants were predominantly female (53.1%), young (M = 22.69 years), and Caucasian (82.3%). Data were analyzed using an actor-partner interdependence model with multilevel structural equation modeling. There were significant actor effects at the between-subjects and within-subjects levels, and significant partner effects for the relationship between PC and social negativity at the within-subject level. Social negativity mediated the relationships between PC and both negative affect and life satisfaction. However, positive affect was more weakly related to PC and social negativity. The social disconnection model was supported. PC was positively associated with one's own social negativity and evoked hostile behaviors from one's partner. Hostile, rejecting behaviors reduced the well-being of the actor, but not the partner. Results suggest perfectionism may be best understood within an interpersonal context.

Neighbourhood social capital: measurement issues and associations with health outcomes

We compared ecometric neighbourhood scores of social capital (contextual variation) to mean neighbourhood scores (individual and contextual variation), using several health-related outcomes (i.e. self-rated health, weight status and obesity-related behaviours). Data were analysed from 5,900 participants in the European SPOTLIGHT survey. Factor analysis of the 13-item social capital scale revealed two social capital constructs: social networks and social cohesion. The associations of ecometric and mean neighbourhood-level scores of these constructs with self-rated health, weight status and obesity-related behaviours were analysed using multilevel regression analyses, adjusted for key covariates. Analyses using ecometric and mean neighbourhood scores, but not mean neighbourhood scores adjusted for individual scores, yielded similar regression coefficients. Higher levels of social network and social cohesion were not only associated with better self-rated health, lower odds of obesity and higher fruit consumption, but also with prolonged sitting and less transport-related physical activity. Only associations with transport-related physical activity and sedentary behaviours were associated with mean neighbourhood scores adjusted for individual scores. As analyses using ecometric scores generated the same results as using mean neighbourhood scores, but different results when using mean neighbourhood scores adjusted for individual scores, this suggests that the theoretical advantage of the ecometric approach (i.e. teasing out individual and contextual variation) may not be achieved in practice. The different operationalisations of social network and social cohesion were associated with several health outcomes, but the constructs that appeared to represent the contextual variation best were only associated with two of the outcomes.

Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all

Cronbach's alpha is a commonly reported estimate to assess scale quality in health psychology and related disciplines. In this paper, we argue that alpha is an inadequate estimate for both validity and reliability - two key elements of scale quality. Omega is a readily available alternative that can be used for both interval and ordinal data. More importantly, we argue that factor-analytic evidence should be presented before assessing the internal structure of a scale. Finally, pointers for readers and reviewers of manuscripts on making judgements about scale quality are provided and illustrated by examples from the field of health psychology.

Detecting short-term change and variation in health-related quality of life: within- and between-person factor structure of the SF-36 health survey

Background

A major goal of much aging-related research and geriatric medicine is to identify early changes in health and functioning before serious limitations develop. To this end, regular collection of patient-reported outcome measure (PROMs) in a clinical setting may be useful to identify and monitor these changes. However, existing PROMs were not designed for repeated administration and are more commonly used as one-time screening tools; as such, their ability to detect variation and measurement properties when administered repeatedly remain unknown. In this study we evaluated the potential of the RAND SF-36 Health Survey as a repeated-use PROM by examining its measurement properties when modified for administration over multiple occasions.

Methods

To distinguish between-person (i.e., average) from within-person (i.e., occasion) levels, the SF-36 Health Survey was completed by a sample of older adults (N = 122, M_age = 66.28 years) daily for seven consecutive days. Multilevel confirmatory factor analysis (CFA) was employed to investigate the factor structure at both levels for two- and eight-factor solutions.

Results

Multilevel CFA models revealed that the correlated eight-factor solution provided better model fit than the two-factor solution at both the between-person and within-person levels. Overall model fit for the SF-36 Health Survey administered daily was not substantially different from standard survey administration, though both were below optimal levels as reported in the literature. However, individual subscales did demonstrate good reliability.

Conclusions

Many of the subscales of the modified SF-36 for repeated daily assessment were found to be sufficiently reliable for use in repeated measurement designs incorporating PROMs, though the overall scale may not be optimal. We encourage future work to investigate the utility of the subscales in specific contexts, as well as the measurement properties of other existing PROMs when administered in a repeated measures design. The development and integration of new measures for this purpose may ultimately be necessary.

Examining the Dynamic Structure of Daily Internalizing and Externalizing Behavior at Multiple Levels of Analysis

Psychiatric diagnostic covariation suggests that the underlying structure of psychopathology is not one of circumscribed disorders. Quantitative modeling of individual differences in diagnostic patterns has uncovered several broad domains of mental disorder liability, of which the Internalizing and Externalizing spectra have garnered the greatest support. These dimensions have generally been estimated from lifetime or past-year comorbidity patters, which are distal from the covariation of symptoms and maladaptive behavior that ebb and flow in daily life. In this study, structural models are applied to daily diary data (Median = 94 days) of maladaptive behaviors collected from a sample (N = 101) of individuals diagnosed with personality disorders (PDs). Using multilevel and unified structural equation modeling, between-person, within-person, and person-specific structures were estimated from 16 behaviors that are encompassed by the Internalizing and Externalizing spectra. At the between-person level (i.e., individual differences in average endorsement across days) we found support for a two-factor Internalizing-Externalizing model, which exhibits significant associations with corresponding diagnostic spectra. At the within-person level (i.e., dynamic covariation among daily behavior pooled across individuals) we found support for a more differentiated, four-factor, Negative Affect-Detachment-Hostility-Disinhibition structure. Finally, we demonstrate that the person-specific structures of associations between these four domains are highly idiosyncratic.

Healthcare service quality: what really matters to the female patient?

Purpose

– This paper aims to develop a model that encompasses the constructs and sub-constructs consumers use in evaluating healthcare service quality (HSQ) in Egypt.

Design/methodology/approach

– Factor analysis was performed on 40 variables to identify the constructs. Ordinal logistic regression was also used to identify the sub-constructs and examine the effect of each sub-construct on patients’ overall perception of service quality.

Findings

– Factor analysis confirmed an eight-construct framework: hospital premises and employees; doctor medical service; nursing medical service; diagnostic medical service; admission; discharge; rooms and housekeeping; and meals. Ordinal logistic regression established 17 sub-constructs – physician reliability; physician assurance; physician interaction; physician’s competence; nursing tangibles; nursing reliability; nursing assurance; nursing interaction; nursing responsiveness; diagnostic service competence; diagnostic service reliability; hospital premises and employees tangibles; admission responsiveness; admission knowledge and courtesy; meals tangibles; rooms tangibles and housekeeping courtesy; and discharge knowledge and courtesy – that have significant effect on HSQ. Some sub-constructs had a significantly greater impact on overall perception of service quality than others.

Practical implications

– Healthcare providers will be able to pinpoint areas of service quality shortfall and better satisfy their patients. This will ultimately lead to repeat patronage and positive recommendation behavior.

Originality/value

– The model is the first comprehensive model in the Middle East that takes into account all constructs and sub-constructs patients use for evaluation of HSQ.

Detecting Intervention Effects in a Cluster-Randomized Design Using Multilevel Structural Equation Modeling for Binary Responses

Multilevel modeling (MLM) is frequently used to detect group differences, such as an intervention effect in a pre-test-post-test cluster-randomized design. Group differences on the post-test scores are detected by controlling for pre-test scores as a proxy variable for unobserved factors that predict future attributes. The pre-test and post-test scores that are most often used in MLM are summed item responses (or total scores). In prior research, there have been concerns regarding measurement error in the use of total scores in using MLM. To correct for measurement error in the covariate and outcome, a theoretical justification for the use of multilevel structural equation modeling (MSEM) has been established. However, MSEM for binary responses has not been widely applied to detect intervention effects (group differences) in intervention studies. In this article, the use of MSEM for intervention studies is demonstrated and the performance of MSEM is evaluated via a simulation study. Furthermore, the consequences of using MLM instead of MSEM are shown in detecting group differences. Results of the simulation study showed that MSEM performed adequately as the number of clusters, cluster size, and intraclass correlation increased and outperformed MLM for the detection of group differences.

Multilevel multidimensional item response model with a multilevel latent covariate

In a pre-test-post-test cluster randomized trial, one of the methods commonly used to detect an intervention effect involves controlling pre-test scores and other related covariates while estimating an intervention effect at post-test. In many applications in education, the total post-test and pre-test scores, ignoring measurement error, are used as response variable and covariate, respectively, to estimate the intervention effect. However, these test scores are frequently subject to measurement error, and statistical inferences based on the model ignoring measurement error can yield a biased estimate of the intervention effect. When multiple domains exist in test data, it is sometimes more informative to detect the intervention effect for each domain than for the entire test. This paper presents applications of the multilevel multidimensional item response model with measurement error adjustments in a response variable and a covariate to estimate the intervention effect for each domain.

The curvilinear relationship between work pressure and momentary task performance: the role of state and trait core self-evaluations

Whereas several studies have demonstrated that core self-evaluations (CSE)–or one’s appraisals about one’s own self-worth, capabilities, and competences–relate to job outcomes, less is known about the mechanisms underlying these relationships. In the present study, we address this issue by examining the role of within- and between-person variation in CSE in the relationship between work pressure and task performance. We hypothesized that (a) work pressure relates to task performance in a curvilinear way, (b) state CSE mediates the curvilinear relationship between work pressure and task performance, and (c) the relationship between work pressure and state CSE is moderated by trait CSE. Our hypotheses were tested via a 10-day daily diary study with 55 employees in which trait CSE was measured at baseline, while work pressure, task performance, and state CSE were assessed on a daily basis. Bayesian multilevel path analysis showed that work pressure affects task performance via state CSE, with state CSE increasing as long as the employee feels that (s)he is able to handle the work pressure, while it decreases when the level of work pressure exceeds the employees’ coping abilities. Moreover, we found that for people low on trait CSE, the depleting effect of work pressure via state CSE happens for low levels of work pressure, while for people high in trait CSE the depleting effect is located at high levels of work pressure. Together, our findings suggest that the impact of work pressure on task performance is driven by a complex interplay of between- and within-person differences in CSE.

Use of weekly assessment data to enhance evaluation of a subjective wellbeing intervention

PURPOSE

While intervention effects in target outcomes have typically been tested based on change from baseline to post-intervention, such approaches typically ignore individual differences in change, including time taken to see improvement. The present study demonstrates how weekly patient-reported data may be used to augment traditional pre-post intervention evaluations in order to gain greater insights into treatment efficacy.

METHODS

Two hundred and fifty-two adolescent boys and girls (M age = 13.6 years, SD = 0.6 years) from four secondary schools in Victoria, Australia, were assigned by school into control (n = 88) or intervention (n = 164) groups. The intervention group participated in a 6-week course designed to improve subjective wellbeing (SWB) by fostering resilience, coping skills, and self-esteem. In addition to baseline, post-intervention, and 3-month follow-up assessments of SWB, intervention group participants also completed weekly summarise of affective experiences for the duration of the intervention phase.

RESULTS

While standard pre-post data showed significant improvement in SWB for the intervention group relative to controls, weekly data showed individual differences in the trajectory of change during this intervention phase; low SWB individuals experienced initial worsening of symptoms followed by improvement in the second half of the intervention phase, whereas high SWB individuals experienced initial gains, followed by a plateau from Week 4 onwards.

CONCLUSIONS

Addition of weekly data provided greater insights into intervention effects by: (1) contradicting the notion that early responsiveness to treatment is predictive of level of improvement by post-intervention, and (2) providing data-based insights into ways to enhance the intervention.

Measuring adaptive performance in individuals and teams

Purpose

– While scales were developed to measure individual adaptive performance (IAP), fewer contributions have been done to assess the construct at the team level of analysis. This issue is addressed through two related studies: Study 1 builds on Pulakos et al. (2000) to develop a measure of IAP. Study 2 follows from the results in Study 1 and tests a measure of team adaptive performance (Chan, 1998).

Design/methodology/approach

– Scale development was done adopting a single level (Study 1) and multi-level (Study 2) structural equations modeling approach.

Findings

– Results suggest that both measures of individual and team adaptive performance are reliable and show evidence supporting the adequacy of adopting referent-shift methodologies to the measurement and aggregation of team members’ rating of team adaptive performance.

Originality/value

– The study offers a reliable, parsimonious and easy to apply measure of individual and team adaptive performance in organizational work environments.

Does body satisfaction influence self-esteem in adolescents' daily lives? An experience sampling study

This study examined, within the context of the Contingencies of Self-Worth model, state-based associations between self-esteem and body satisfaction using the experience sampling method. One hundred and forty-four adolescent girls (mean age = 14.28 years) completed up to 6 assessments per day for one week using Palm Digital Assistants, in addition to baseline measures of trait body satisfaction and self-esteem. Results showed considerable variation in both state-based constructs within days, and evidence of effects of body satisfaction on self-esteem, but not vice versa. Although these state-based associations were small in size and weakened as the time lag between assessments increased for the sample as a whole, individual differences in the magnitude of these effects were observed and predicted by trait self-esteem and body satisfaction. Collectively, these findings offer support for key tenets of the Contingencies of Self-Worth model.

Multilevel Latent Polynomial Regression for Modeling (In)Congruence Across Organizational Groups

This article addresses (in)congruence across different kinds of organizational respondents or “organizational groups”—such as managers versus non-managers or women versus men—and the effects of congruence on organizational outcomes. We introduce a novel multilevel latent polynomial regression model (MLPM) that treats standings of organizational groups as latent “random intercepts” at the organization level while subjecting these to latent interactions that enable response surface modeling to test congruence hypotheses. We focus on the case of organizational culture research, which usually samples managers and excludes non-managers. Reanalyzing data from 67 hospitals with 6,731 managers and non-managers, we find that non-managers perceive their organizations’ cultures as less humanistic and innovative and more controlling than managers, and we find that less congruence between managers and non-managers in these perceptions is associated with lower levels of quality improvement in organizations. Our results call into question the validity of findings from organizational culture and other research that tends to sample one organizational group to the exclusion of others. We discuss our findings and the MLPM, which can be extended to estimate latent interactions for tests of multilevel moderation/interactions.

Conceptualizing and Measuring Self-Criticism as Both a Personality Trait and a Personality State

Blatt's ( 2004 , 2008 ) conceptualization of self-criticism is consistent with a state-trait model that postulates meaningful variation in self-criticism both between persons (traits) and within person (states). We tested the state-trait model in a 7-day diary study with 99 college student participants. Each evening they completed a 6-item measure of self-criticism, as well as measures of perceived social support, positive and negative affect, compassionate and self-image goals during interactions with others, and interpersonal behavior, including overt self-criticism and given social support. As predicted, self-criticism displayed both trait-like variance between persons and daily fluctuations around individuals' mean scores for the week; slightly more than half of the total variance was between persons (ICC = .56). Numerous associations at both the between-persons and within-person levels were found between self-criticism and the other variables, indicating that individuals' mean levels of self-criticism over the week, and level of self-criticism on a given day relative to their personal mean, were related to their cognitions, affect, interpersonal goals, and behavior. The results supported the construct validity of the daily self-criticism measure. Moreover, the findings were consistent with the state-trait model and with Blatt's theoretical analysis of self-critical personality.

Teaching Through Interactions in Secondary School Classrooms: Revisiting the Factor Structure and Practical Application of the Classroom Assessment Scoring System-Secondary

Valid measurement of how students' experiences in secondary school classrooms lead to gains in learning requires a developmental approach to conceptualizing classroom processes. This article presents a potentially useful theoretical model, the Teaching Through Interactions framework, which posits teacher-student interactions as a central driver for student learning and that teacher-student interactions can be organized into three major domains. Results from 1,482 classrooms provide evidence for distinct emotional, organizational, and instructional domains of teacher-student interaction. It also appears that a three-factor structure is a better fit to observational data than alternative one- and two-domain models of teacher-student classroom interactions, and that the three-domain structure is generalizable from 6th through 12th grade. Implications for practitioners, stakeholders, and researchers are discussed.

Felt Security in Daily Interactions as a Mediator of the Effect of Attachment on Relationship Satisfaction†Some of the Results Reported in this Article Were Previously Presented at the Annual Meeting of the Society for Interpersonal Theory and Research in Montreal, Canada (May 2012)

This study examined how felt security in interpersonal situations with one's romantic partner mediated the effect of global (dispositional) attachment on relationship satisfaction. Felt security was measured using an event–contingent recording (ECR) methodology with a sample of 93 cohabiting couples who reported their social interactions with each other during a 20–day period. Global attachment was measured at the beginning of the ECR procedure. Relationship satisfaction was measured at the end of the ECR procedure (T1) and approximately 7 months after the ECR procedure (T2). Results confirmed the established links between attachment and relationship satisfaction such that higher attachment avoidance and attachment anxiety were associated with decline in satisfaction over time. Results also indicated that attachment avoidance but not attachment anxiety was negatively related to felt security, both within–partner and across–partners. As expected, lower felt security exerted a negative effect on relationship satisfaction at T1 and T2, and partly mediated the effect of attachment avoidance on relationship satisfaction at T1 and T2, both within–partner and across–partners. Partners’ gender emerged as a moderator of these results. Findings suggest higher attachment avoidance leads to less felt security in daily social interactions, which leads to less satisfaction with the romantic relationship. Copyright © 2015 European Association of Personality Psychology