Applying Item Response Theory (IRT) Modeling to an Observational Measure of Childhood Pragmatics: The Pragmatics Observational Measure-2

Evaluation of a New Adolescent Social Communication Assessment: The Transition Pragmatics Interview

PURPOSE

This study evaluated the psychometric functioning of a new criterion-referenced assessment of adolescent social communication, the Transition Pragmatics Interview (TPI), based on the synthesis model of pragmatics. Two ways of interpreting item difficulty were explored: (a) as a function of the synthesis model elements of social communication ability that items were designed to assess, and (b) as a function of the developmental level required for a successful response based on an adapted situational-discourse-semantics (SDS) model (Norris & Hoffman, 1993).

METHOD

Thirty-seven participants aged 14-22 years completed the TPI. Responses were analyzed using Rasch analysis to evaluate the functioning of the scale and to determine item difficulty. Items were coded for the SDS developmental level required for an adequate response. The mean Rasch item difficulty for items at each SDS developmental level was analyzed for the five adapted SDS domains.

RESULTS

Consistent with the first approach for interpreting item difficulty, TPI items varied in difficulty as a function of the element of social communication they were designed to assess (p < .001). Interpreting item difficulty based on the adapted SDS model was not supported: Items requiring higher SDS developmental levels were not more difficult than those requiring less (p = .55).

CONCLUSIONS

The TPI responses fit the Rasch model, supporting the TPI as a unidimensional measure and supporting the use of all items together to compute a single number that summarizes the level of social communication for each examinee. The item ordering from least to most difficult was consistent with prior findings on adolescent social communication development.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.26018545.

Validation of the Korean Version of Culturally Responsive Experiences in Close Relationships-Short Form

The authors developed and validated the Korean version of the Experiences in Close Relationships-Short Form (K-ECRR-SF) with the goal of developing a culturally responsive scale. In study 1, a Rasch analysis was conducted on the 36 original items in the ECR-Revised (ECR-R) to select items that best represent anxiety and avoidance subscales by considering cultural equivalence. In study 2, confirmatory factor analysis (CFA) was conducted for the selected 12 items with a different sample. The factor structures of the ECR-R and K-ECRR-SF through CFA were then compared through CFA. In addition, the K-ECRR-SF items were tested for related constructs (i.e., reassurance and support seeking, loneliness, dyadic satisfaction, depression, anxiety, and fear of intimacy) to its criterion evidence. The newly developed K-ECRR-SF is confirmed to be valid and culturally responsive scale in measuring attachment in Korea.

Application of Item Response Theory (IRT)-Graded Response Model (GRM) to Entrepreneurial Ecosystem Scale

The scale of entrepreneurial ecosystems (EE) assesses the perceptions about entrepreneurial ecosystem domains, finances, capital finances, support, support professions, policies, markets, human resources, and culture. The scales are always error-prone—these scales must possess properties that enable them it to provide maximum information and validity reliability. Convenient sampling data from (n = 474) founders, co-founders, and entrepreneurs were collected. The IRT-GRM model is used to validate and test the instrument-based on polytomous scales. IRT yields discriminating power—the level of difficulty of the items of the scale. The scale consists of 48 items. The item Pol5 (4.13) was found to have the highest discriminating value (4.13), the item mar5 had the lowest discriminating value (1.57), and all items had discriminating values greater than the threshold value of 0.60. The EE Scale showed good reliability based on McDonald’s omega and Cronbach’s alpha (0.80 and 0.88). The parallel and factor analysis showed good agreement of the one-dimesnionality of the scale. The model goodness of fit statistics based on the comparative fit index (CFI) and the Tucker–Lewis index, (TLI) and the standardized root mean square residual (SRMR) showed a satisfactory level of fit; however, the root mean square error of approximation (RMSE) showed a poor fit. The item characteristic curves showed that the all item responses were properly ordered. The items of the scale showed a satisfactory level of discrimination power and level of difficulty, and it was found to have three levels of agreement about entrepreneurial ecosystem scale. It is concluded that the EE scale possesses good psychometric properties and that it is reliable and valid instrument to measure the entrepreneurial ecosystem of the given region.

Peabody Developmental Motor Scales-2: The Use of Rasch Analysis to Examine the Model Unidimensionality, Motor Function, and Item Difficulty

The Peabody Developmental Motor Scales-Second Edition (PDMS-2) is a valid and reliable instrument used in several countries, including Brazil, to assess gross and fine motor skills and identify motor deficits and eligibility for intervention for children with and without disabilities. However, the analysis of PDMS-2 items regarding the unidimensionality of the model, order of item difficulty, and whether the items portray the children's developmental trajectories still lacks investigation. Therefore, this study aims to: (1) analyze the unidimensionality of PDMS-2, (2) verify the model's capacity to explain the variance in the motor function responses, and (3) identify the level of difficulty of the items for Brazilian children. Children (n = 637; 51% girls) newborn to 71 months (M age = 21.7, SD = 18.6) were assessed using the PDMS-2. The Rasch analysis was conducted; the indexes of infit and outfit, and the point-biserial correlations coefficient were analyzed. The model unidimensionality was investigated using percentages of variance in the Rasch model (40% of variance). Results indicated that (1) for reflexes subscale, 62.5% of the items had correlations with the factor above 0.60, and two items had unadjusted infit and outfit; (2) for stationary subscale, 83.3% of the correlations of the items with the factor were above 0.50, and one item had unadjusted infit and outfit; (3) for locomotion subscale, 80.0% of the correlation of the items with the factor were above 0.50; all items had adequate infit and outfit; (4) for object manipulation subscale, 79.9% of the correlation of the items with the factor were above 0.50, and one item had unadjusted infit and outfit; (5) for grasping subscale, 92.3% of the correlation of the items with the factor were above 0.50, and one item had unadjusted infit and outfit; and (6) for the visual-motor integration subscale, 73.6% of the correlation of the items with the factor were above 0.50, and six items had unadjusted infit and outfit. The items with unadjusted fit were removed for further analysis. No changes in reliability and separation of items and people scores were observed without the unadjusted items; therefore, all items were maintained. A unidimensional model was found, and the reliability and discriminant capability of the items were adequate, and all items should be used to assess children. The PDMS-2 is appropriate for assessing Brazilian children.

A twelve-month follow-up of an information communication technology delivered intervention for children with autism spectrum disorder living in regional Australia

This study investigated the long-term follow-up of an information communication techonology based intervention, the Therapeutic Outcomes By You application, for children with autism spectrum disorder living in regional Australia. Fifteen participants who completed a three-month randomised controlled trial of the Therapeutic Outcomes By You were assessed at least 12 months post-intervention to determine the maintenance or continued improvement of their language and social communication skills. Findings demonstrate the receptive language, social skills, pragmatic language and playfulness of children with autism spectrum disorder improved during the three-month intervention period and were maintained at least 12 months after ceasing the Therapeutic Outcomes By You app intervention.

A Play-Based, Peer-Mediated Pragmatic Language Intervention for School-Aged Children on the Autism Spectrum: Predicting Who Benefits Most

This study explored characteristics of children with autism with large intervention effects following a peer-mediated pragmatic language intervention, to devise algorithms for predicting children most likely to benefit. Children attended a 10-week intervention with a typically-developing peer. Data from a pilot study and RCT formed the dataset for this study. The POM-2 measured intervention outcomes. Children completed the EVT-2, TACL-4, and Social Emotional Evaluation at baseline, and parents completed the CCC-2 and CCBRS. High CCC-2 Use of Context and CCBRS Separation Anxiety scores and comparatively lower EVT-2, CCC-2 Nonverbal Communication and Cohesion scores predicted children with large intervention effects. Results can be used by clinicians to predict which children within their clinics might benefit most from participating in this intervention.

A Randomized Controlled Trial of a Play-Based, Peer-Mediated Pragmatic Language Intervention for Children With Autism

PURPOSE

This randomized controlled trial evaluated the effectiveness of a play-based pragmatic language intervention for children with autism.

METHODS

A sample of 71 children with autism were randomized to an intervention-first group (n = 28 analyzed) or waitlist-first (n = 34 analyzed) group. Children attended 10, weekly clinic play-sessions with a typically developing peer, and parents mediated practice components at home. The Pragmatics Observational Measure (POM-2) and the Social Emotional Evaluation (SEE) evaluated pragmatics before, after and 3-months following the intervention.

RESULTS

POM-2 gains were greatest for intervention-first participants (p = 0.031, d = 0.57). Treatment effects were maintained at 3-month follow-up (p < 0.001-0.05, d = 0.49-0.64). POM-2 scores were not significantly different in the clinic and home settings at follow-up.

CONCLUSION

Findings support the combination of play, peer-mediation, video-feedback and parent training to enhance pragmatic language in children with autism.