A contextual study of daily living strategies in neurocognitively impaired adults with schizophrenia

Comparison of performance-based assessment and real world skill in people with serious mental illness: Ecological validity of the Test of Grocery Shopping Skills

Valid functional measures are essential for clinical and research efforts that address recovery and community functioning in people with serious mental illness. Although there is a great deal of interest in functional assessment, there is limited research supporting how well current evaluation methods provide a true assessment of real world functioning or naturalistic behavior. To address this gap in the literature, the present study examined the performance of individuals with serious mental illness (i.e., diagnosis of schizophrenia-spectrum, bipolar disorder, or other depression/anxiety diagnoses and accompanying functional disability) on the Test of Grocery Shopping Skills (TOGSS), a performance-based naturalistic task. We compared TOGSS performance to two dimensions of real world functioning: directly observed real world grocery shopping and ratings of community functioning. Results indicated that the TOGSS was significantly associated with real life grocery shopping, in terms of both shopping accuracy (r = 0.424) and time (r = 0.491). Further, self-report and observer-rated methods of assessing real world shopping behaviors were significantly correlated (r = 0.455). To our knowledge, this is one of the first studies to directly compare a performance-based naturalistic skill assessment with carefully observed real world performance of that skill in people with serious mental illness. These findings support the feasibility and ecological validity of performance-based naturalistic assessment with the TOGSS.

A video ethnography approach to assessing the ecological validity of neurocognitive and functional measures in severe mental illness: results from a feasibility study

BACKGROUND

The ecological validity of neurocognitive and functional measures in severe mental illness is poorly understood because of a lack of validated research methods to study community life-as-lived. We describe the development of a video ethnography method that measures naturalistic behaviors with codes called community performance indicators (CPIs). The method could provide a strategy to test the ecological validity of neurocognitive and functional assessments.

METHODS

We gathered up to 18.5 hours of video ethnography data on each of 9 subjects with schizophrenia selected for high or low composite scores on the MATRICS Consensus Cognitive Battery (MCCB). We used video ethnography to capture subjects' everyday behaviors in their usual environments. We established 4 CPIs that showed excellent inter-rater and promising test-retest reliability: (1) behavioral activity level, (2) goal pursuit, (3) social interaction, and (4) problem solving.

RESULTS

(1) High and low MCCB subjects showed statistically significantly differences on all 4 CPIs. (2) MCCB composite scores were correlated with all 4 CPIs (r = .54 to -.77, P < .01 to .07). (3) The MCCB domain scores demonstrated some specificity in their correlations with the CPIs; eg, verbal learning, reasoning/problem solving, and social cognition were correlated with CPI domains of social interaction and problem solving.

CONCLUSIONS

We present a method for reliably measuring everyday functional performance in schizophrenia. Results from a small select sample suggest that CPIs capture skills associated with neurocognition, supporting their use in a larger study of ecological validity.

A video ethnography approach for linking naturalistic behaviors to research constructs of neurocognition in schizophrenia

Few methods are available to explore the impact of neurocognition in schizophrenia on behaviors performed in usual contexts. The authors developed a video ethnography approach to examine the relationship between naturalistic behaviors and research constructs of neurocognition. Video ethnographers accompanied subjects through usual routines gathering continuous video data. Researchers developed codes to measure four behavioral domains observed on video. This paper describes the psychometric characteristics to be considered in the development of observational approaches. It also highlights differences between behaviors performed in usual environments and neuropsychological constructs. The authors demonstrate that everyday behaviors that have been shown to correspond to neurocognitive skills in a pilot feasibility study can be identified and rated. They further suggest that observational methods could provide novel strategies for linking research findings and clinical concerns.

Assessing function and functional outcome in schizophrenia

The diagnosis of schizophrenia can only be made in the presence of a loss of functioning in domains such as employment, independent living, and social functioning. Accurately measuring functioning is central to research on the course of the disorder, treatment and rehabilitation outcomes, and biosocial factors in schizophrenia. Assessments of functional disability have described three dimensions of functioning: functional capacity, functional performance, and functional outcome. The "competence/performance" distinction refers to the observation that an individual may demonstrate an ability to perform a functional task (capacity) but may not do so in her own community environment (performance). Functional outcomes are the result of both capacity and performance. Several recent reviews have compared the characteristics, reliability, and validity of various functional assessment instruments. Two major initiatives are underway to gather additional comparative data about functional assessment strategies. Recently, both the recovery movement and the recognition of the role of environmental factors in functioning have raised questions about the conceptual content of the functioning construct (construct validity). For instance, several studies have demonstrated that features of functioning need not track together over the course of the illness. In addition, the notion of recovery emphasizes processes like community integration and subjective well-being that are not static outcomes but are continually evolving features of the life course in chronic illness. Findings on the dynamic role of environmental moderators such as support and opportunity also present challenges to scientific constructs. For these reasons and others, the ecological validity of functional assessments has become a central concern. Both the verisimilitude and veridicality of functional assessments can be empirically assessed, but to date very few studies have measured the extent to which functional measures accurately predict individuals' behavior in their usual environments. Observational studies in naturalistic environments are one important area for future research.

Understanding the ecological validity of neuropsychological testing using an ethnographic approach

Neurocognitive impairment is a defining and disabling feature of schizophrenia and other physical disorders. Most of our understanding about neurocognitive deficits comes from laboratory-based testing in research protocols. There has been little research using direct behavioral community observation over a prolonged period to understand the association of daily functioning with cognitive performance.The purpose of this study was to develop an observational method that could be replicated by researchers interested in viewing cognitive deficits in vivo, and then comparing this data to laboratory measures to affirm the ecological validity of those measures.The eight-step method explained here was developed from the targeted ethnographic study of 10 persons with schizophrenia. Obtaining real world context with this method will help to increase the generalizability of effective cognitive treatments, create improved interventions for this population, and bring into greater relief the coping and compensatory strategies already used by individuals to complete daily tasks.