Using Scientific Knowledge to Inform Preschool Assessment: Making the Case for “Empirical Validity”

HTKS-Kids: A tablet-based self-regulation measure to equitably assess young children's school readiness

Background

Technology advances make it increasingly possible to adapt direct behavioral assessments for classroom use. This study examined children's scores on HTKS-Kids, a new, largely child-led version of the established individual research assessment of self-regulation, Head-Toes-Knees-Shoulders-Revised task (HTKS-R). For the HTKS-Kids tablet-based assessment, which was facilitated by children's preschool teachers, we examined (1) preliminary reliability and validity; (2) variation in scores predicted by child age and background characteristics; and (3) indication that HTKS-Kids provides different information from teacher ratings of children.

Method

Participants included n = 79 4-year-old children from two urban areas in upstate New York, USA. Average parent education was 12.5 years, ranging 3-20. A researcher administered the HTKS-R to individual children, and teachers (eight white, two Latino) were trained to use the HTKS-Kids tablet-based assessment and asked to play once with each study child. Teachers also rated each child on 10 Child Behavior Rating Scale (CBRS) items about classroom self-regulation.

Results

We found evidence that (1) the HTKS-Kids captures variation in children's self-regulation and correlates positively with established measures, (2) parent education was the best predictor of HTKS-Kids scores, and (3) teachers rated Black children significantly worse and white children better on the CBRS, with the magnitude of group differences similar to the contribution of parent education. In contrast, Black and white children showed no score differences on HTKS-Kids.

Implications

The HTKS-Kids is a promising new tablet-based assessment of self-regulation that could replace or supplement traditional teacher ratings, which are often subject to implicit bias.

The cross-cultural validation of the Beach Center Family Quality of Life Scale among persons affected by leprosy or podoconiosis in Northwest Ethiopia

BACKGROUND

The Beach Center Family Quality of Life Scale has been developed and validated in different languages in different countries. However, this scale has not been validated in the Ethiopian Amharic language context. Therefore, this study aimed to investigate the cross-cultural validity of the Beach Center Family Quality of Life Scale, among Ethiopian families of persons affected by leprosy and podoconiosis.

METHODOLOGY

We explored the semantic equivalence, internal consistency, reproducibility, floor and ceiling effects, and interpretability of the Beach Center Family Quality of Life Scale in Amharic. A cross-sectional study was conducted after the translation and back-translation of the instrument. A total of 302 adult persons affected by leprosy or podoconiosis was asked about their level of satisfaction with their family life, using the Beach Center Family Quality of Life Scale. In addition, 50 participants were re-interviewed two weeks after the initial assessment to test the reproducibility of the scale. Participants were recruited in the East Gojjam zone of Northwest Ethiopia.

RESULTS

The findings of this study showed that the Beach Center Family Quality of Life Scale had high internal consistency (Cronbach's alpha of 0.913) and reproducibility (intra-class correlation coefficient of 0.857). The standard error of measurement was 3.01, which is 2.4% of the total score range. The smallest detectable change was 8.34. Confirmatory factor analysis showed adequate factor loadings and model fit indices like the original scale. The composite reliability and average variance extracted from the scale were acceptable. No floor and ceiling effects were found.

CONCLUSIONS

Our findings indicate that the Amharic version of the Beach Center Family Quality of Life Scale has adequate cultural validity to assess the family quality of life in Ethiopian families of persons affected by leprosy and podoconiosis.

Early childhood education language environments: considerations for research and practice

The importance of developing early language and literacy skills is acknowledged by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) as a global human rights issue. Indeed, research suggests that language abilities are foundational for a host of cognitive, behavioral, and social-emotional outcomes. Therefore, it is critical to provide experiences that foster language acquisition across early learning settings. Central to these efforts is incorporating assessments of language environments into research and practice to drive quality improvement. Yet, several barriers may be preventing language environment assessments from becoming widely integrated into early education. In this brief, we review evidence on the types of experiences that promote language development, describe characteristics of language environment assessments, and outline practical and philosophical considerations to assist with decision-making. Further, we offer recommendations for future research that may contribute knowledge regarding strategies to assess and support language development. In addressing both areas, we highlight the potential for early childhood language environments to advance equity.

Reliability, validity and developmental sensitivity of the Language Use Inventory (LUI) in the Chinese context

BACKGROUND

Pragmatics has generally been defined as the ability to use language in social situations, it is commonly regarded as the third major component of language ability. To date, there is no tool for assessing early pragmatic development of Chinese-speaking children.

AIMS

To describe the translation of the Language Use Inventory (LUI) from English to Mandarin Chinese and to report findings on the Chinese version's reliability, validity and developmental sensitivity.

METHODS & PROCEDURES

The original English version of the LUI was translated into Mandarin Chinese. Parents of 177 typically developing (TD) toddlers and preschool children completed the inventory to examine its internal reliability and construct validity and how scores differed across ages and sexes. A total of 31 parents out of the 177 completed the LUI-Mandarin, again within 4 weeks, to assess test-retest reliability. To examine discriminative validity, 43 parents of age- and sex-matched TD children and children with autism spectrum disorder (ASD) recruited from Nanjing Brain Hospital affiliated with Nanjing Medical University completed the LUI-Mandarin.

OUTCOMES & RESULTS

Cronbach's alpha values for the LUI-Mandarin's three parts and for 11 of 12 LUI-Mandarin subscales were 0.707-0.992, with most values in the 0.825-0.992 range. Test-retest reliability ranged from 0.66 to 0.95, indicating good to excellent reliability. Factor analysis of the LUI-Mandarin revealed two different factors, and the total variance explained was 74.38%. The LUI-Mandarin total scores and subscale scores increased with age for both boys and girls, providing evidence of the inventory's developmental sensitivity. Girls, however, had higher total scores than boys at earlier ages (18-23 months). The results of the discriminant validity study revealed that performance was significantly lower in the ASD group than in the TD group with respect to LUI total scores and subscale scores (except for subscale A).

CONCLUSIONS & IMPLICATIONS

The LUI-Mandarin is the first and only questionnaire available in China that evaluates the pragmatic language skills of children aged between 18 and 47 months. The results of the study show that the LUI-Mandarin is a valid and reliable tool for Chinese toddlers and preschool children.

WHAT THIS PAPER ADDS

What is already known on this subject The LUI is a parent-report questionnaire that can provide comprehensive information about very young children's communicative competence. It is widely used both for assessment and to guide intervention. Additionally, it has been translated into French, Italian, Polish, Arabic, Portuguese and Norwegian and it shows good reliability and validity. What this paper adds to existing knowledge In the present study we describe the translation of the LUI from English to Mandarin Chinese and report findings on the Chinese version's reliability, validity and developmental sensitivity. What are the potential or actual clinical implications of this work? The LUI-Mandarin is the first and only questionnaire available in China that can evaluate pragmatic language skills of children aged between 18 and 47 months. The results show that the LUI-Mandarin is a valid and reliable tool for use with Chinese toddlers and preschool children.

Emerging School Readiness Profiles: Motor Skills Matter for Cognitive- and Non-cognitive First Grade School Outcomes

A promising approach for studying school readiness involves a person-centered approach, aimed at exploring how functioning in diverse developmental domains conjointly affects children’s school outcomes. Currently, however, a systematic understanding lacks of how motor skills, in conjunction with other school readiness skills, affect a child’s school outcomes. Additionally, little is known about longitudinal associations of school readiness with non-academic (e.g., socioemotional) school outcomes. Therefore, we examined the school readiness skills of a sample of Dutch children (N = 91) with a mean age of 3 years and 4 months (46% girls). We used a multi-informant test battery to assess children’s school readiness in terms of executive functions (EFs), language and emergent literacy, motor skills, and socioemotional behavior. During the spring term of a child’s first grade year, we collected academic and non-academic (i.e., EFs, motor skills, socioemotional- and classroom behavior, and creative thinking) school outcomes. A latent profile analysis revealed four distinct profiles. Children in the “Parent Positive” (29%) profile were rated positively by their parents, and performed variably on motor and language/emergent literacy skills tests. The second profile–“Multiple Strengths” (13%)–consisted of children showing strengths in multiple domains, especially with respect to motor skills. Children from the third profile–“Average Performers” (50%)–did not show any distinct strengths or weaknesses, rather displayed school readiness skill levels close to, or just below the sample mean. Finally, the “Parental Concern” (8%) profile was characterized by high levels of parental concerns, while displaying slightly above average performance on specific motor and language skills. Motor skills clearly distinguished between profiles, next to parent-rated EFs and socioemotional behavior, and to a lesser extent emergent literacy skills. School readiness profiles were found to differ in mean scores on first grade academic achievement, parent- and teacher-rated EFs, motor skills, parent-rated socioemotional functioning, and pre-requisite learning skills. The pattern of mean differences was complex, suggesting that profiles could not be ranked from low to high in terms of school outcomes. Longitudinal studies are needed to disentangle the interaction between emerging school readiness of the child and the surrounding context.

Research to Establish the Validity, Reliability, and Clinical Utility of a Comprehensive Language Assessment of Mandarin

PURPOSE

With no existing gold standard for comparison, challenges arise for establishing the validity of a new standardized Mandarin language assessment normed in mainland China.

METHOD

A new assessment, Diagnostic Receptive and Expressive Assessment of Mandarin (DREAM), was normed with a stratified sample of 969 children ages 2;6 (years;months) to 7;11 in multiple urban and nonurban regions in northern and southern China. In this study of 230 children, the sensitivity and specificity of DREAM were examined against an a priori judgment of disorders. External validity was assessed using 2 indices of language production for different age groups.

RESULTS

External validity was assessed against spontaneous language indices (correlation range: r = .6-.7; all ps < .01) and narrative indices (overall: r = .45, p < .01). Sensitivity (.73) and specificity (.82) of DREAM are moderate to good using a priori judgment as the standard. The values improved to .95 and .82 when spontaneous language and narratives were added to a priori judgment to define typicality. Divergent validity was moderate with nonlinguistic indices.

CONCLUSION

DREAM holds promise as a diagnostic test of Mandarin language impairment for children aged 2;6 to 7;11.

Deconstructing building blocks: preschoolers' spatial assembly performance relates to early mathematical skills

This study focuses on three main goals: First, 3-year-olds' spatial assembly skills are probed using interlocking block constructions (N = 102). A detailed scoring scheme provides insight into early spatial processing and offers information beyond a basic accuracy score. Second, the relation of spatial assembly to early mathematical skills was evaluated. Spatial skill independently predicted a significant amount of the variability in concurrent mathematical performance. Finally, the relation between spatial assembly skill and socioeconomic status (SES), gender, and parent-reported spatial language was examined. While children's performance did not differ by gender, lower SES children were already lagging behind higher SES children in block assembly. Furthermore, lower SES parents reported using significantly fewer spatial words with their children.

Using Teacher Ratings to Track the Growth and Development of Young Children Using the Teaching Strategies GOLD® Assessment System

An important consideration in determining the validity of an observational assessment measure for young children is the variability attributed to the child versus that ascribed to the assessor or to some other factor such as classroom context. The Teaching Strategies GOLD® assessment system was used to elicit teacher ratings of a national sample of 21,592 children (age 12-51 months). Teacher ratings of child development and learning were associated in expected directions with both child demographic characteristics and classroom composition variables. Children with disabilities started behind their typically developing peers and grew slower, girls showed an advantage in some areas over boys, and English language learners (ELLs) were rated lower at the beginning of the year and showed some faster rates of growth than their native English-speaking peers.

The language use inventory for young children: a parent-report measure of pragmatic language development for 18- to 47-month-old children

PURPOSE

To demonstrate the internal reliability and discriminative validity of the Language Use Inventory for Young Children (LUI; D. K. O'Neill, 2002), a newly developed parent-report measure designed to assess pragmatic language development in 18-47-month-olds.

METHOD

To examine internal reliability, the LUI was completed by mail by 177 parents recruited from the University of Waterloo's Centre for Child Studies database, 175 of whom completed the LUI again within 4 weeks to assess test-retest reliability. To examine discriminative validity, 49 parents of children awaiting assessment at a local speech-language clinic and 49 parents of typically developing children recruited from the Centre for Child Studies database and matched in age and sex to the clinic group completed the LUI.

RESULTS

Alpha values for the subscales of the LUI were at or above acceptable levels (.80-.98), and steady growth in children's pragmatic language development was demonstrated. The study of discriminant validity revealed sensitivity and specificity levels over 95%.

CONCLUSIONS

The LUI's internal reliability and stability were strongly supported and its sensitivity and specificity in distinguishing between typically developing and language-delayed children exceeded even the most stringent criteria of 90% accuracy.

Pathways to Reading: The Role of Oral Language in the Transition to Reading

What is the role of oral language in reading competence during the transition to school? Is oral language in preschool best conceptualized as vocabulary knowledge or as more comprehensive language including grammar, vocabulary, and semantics? These questions were examined longitudinally using 1,137 children from the National Institute of Child Health and Human Development Study of Early Child Care and Youth Development. Children were followed from age 3 through 3rd grade, and the results suggest that oral language conceptualized broadly plays both a direct and an indirect role in word recognition during the transition to school and serves as a better foundation for early reading skill than does vocabulary alone. Implications of these findings are discussed in terms of both theoretical models of early reading and practical implications for policy and assessment.