An Application of Item Response Theory and the Rasch Model in Speech Recognition Test Materials

A review of Mandarin speech recognition test materials for use in Singapore

OBJECTIVE

This study aimed to evaluate word lists of current Mandarin speech recognition test materials on their adaptability for use on adult Singapore Mandarin speakers.

DESIGN

This scoping review involved literature search through electronic databases for studies published in English or Mandarin, detailing the development or validation of Mandarin speech recognition test materials meant to be used in quiet. Word lists of materials were evaluated for phonemic balance (Stage 1) and familiarity to Singapore Mandarin speakers by comparing the test items against a list of 1,000 common Singapore Mandarin words (Stage 2).

STUDY SAMPLE

Thirty articles that fulfilled the inclusion and exclusion criteria were reviewed.

RESULTS

Nine sets of materials were found and evaluated in Stage 1. Among these, two sets of materials did not contain phonemically balanced word lists. The remaining seven sets of materials were evaluated in Stage 2. The percentage of unfamiliar characters to Singapore Mandarin speakers was found to be between 15% and 41% across the word lists.

CONCLUSION

None of the currently available materials contain word lists that can be readily adapted for re-recording with the voice of a Singapore Mandarin speaker. There is a need to develop original Singapore Mandarin speech recognition test materials.

Applying Rasch model analysis in the development of the cantonese tone identification test (CANTIT)

OBJECTIVE

Applying Rasch analysis to evaluate the internal structure of a lexical tone perception test known as the Cantonese Tone Identification Test (CANTIT).

DESIGN

A 75-item pool (CANTIT-75) with pictures and sound tracks was developed. Respondents were required to make a four-alternative forced choice on each item. A short version of 30 items (CANTIT-30) was developed based on fit statistics, difficulty estimates, and content evaluation. Internal structure was evaluated by fit statistics and Rasch Factor Analysis (RFA).

STUDY SAMPLE

200 children with normal hearing and 141 children with hearing impairment were recruited.

RESULTS

For CANTIT-75, all infit and 97% of outfit values were < 2.0. RFA revealed 40.1% of total variance was explained by the Rasch measure. The first residual component explained 2.5% of total variance in an eigenvalue of 3.1. For CANTIT-30, all infit and outfit values were < 2.0. The Rasch measure explained 38.8% of total variance, the first residual component explained 3.9% of total variance in an eigenvalue of 1.9.

CONCLUSIONS

The Rasch model provides excellent guidance for the development of short forms. Both CANTIT-75 and CANTIT-30 possess satisfactory internal structure as a construct validity evidence in measuring the lexical tone identification ability of the Cantonese speakers.

Using Rasch Analysis to Examine the Item-Level Psychometrics of the Infant-Toddler Meaningful Auditory Integration Scales

The Infant-Toddler Meaningful Auditory Integration Scales (IT-MAIS; Zimmerman-Phillips et al. 2001) is a popular assessment designed to measure listening skills in children with hearing loss aged 0-3 years. For this study we analyzed the item-level psychometric properties of the IT-MAIS via Rasch analysis to gain further understanding about its validity and reliability. We chose to analyze the psychometric properties of the IT-MAIS because very little information exists regarding its development and validation, although it is widely used to assess listening skills in children with SNHL ages 0 to 3 years pre- and post-CI. Our results indicated that the IT-MAIS items demonstrated less than ideal psychometric properties and the IT-MAIS item order did not reflect the order in which children are expected to develop functional listening skills. Our findings suggest that there is a pressing need for further discussion among researchers and clinicians about 1) how the IT-MAIS is used, and 2) what other valid and reliable outcome measures could be used alongside, or in place of, the IT-MAIS to determine CI candidacy, establish treatment goals, or track progress in listening development in very young children with hearing loss.