Mandarin Chinese Tone Recognition with an Artificial Neural Network

Implementation of Machine Learning on Human Frequency-Following Responses: A Tutorial

The frequency-following response (FFR) provides enriched information on how acoustic stimuli are processed in the human brain. Based on recent studies, machine learning techniques have demonstrated great utility in modeling human FFRs. This tutorial focuses on the fundamental principles, algorithmic designs, and custom implementations of several supervised models (linear regression, logistic regression, k -nearest neighbors, support vector machines) and an unsupervised model ( k -means clustering). Other useful machine learning tools (Markov chains, dimensionality reduction, principal components analysis, nonnegative matrix factorization, and neural networks) are discussed as well. Each model's applicability and its pros and cons are explained. The choice of a suitable model is highly dependent on the research question, FFR recordings, target variables, extracted features, and their data types. To promote understanding, an example project implemented in Python is provided, which demonstrates practical usage of several of the discussed models on a sample dataset of six FFR features and a target response label.

Acoustic Assessment of Tone Production of Prelingually-Deafened Mandarin-Speaking Children With Cochlear Implants

Objective

The purpose of the present study was to investigate Mandarin tone production performance of prelingually deafened children with cochlear implants (CIs) using modified acoustic analyses and to evaluate the relationship between demographic factors of those CI children and their tone production ability.

Methods

Two hundred seventy-eight prelingually deafened children with CIs and 173 age-matched normal-hearing (NH) children participated in the study. Thirty-six monosyllabic Mandarin Chinese words were recorded from each subject. The fundamental frequencies (F0) were extracted from the tone tokens. Two acoustic measures (i.e., differentiability and hit rate) were computed based on the F0 onset and offset values (i.e., the tone ellipses of the two-dimensional [2D] method) or the F0 onset, midpoint, and offset values (i.e., the tone ellipsoids of the 3D method). The correlations between the acoustic measures as well as between the methods were performed. The relationship between demographic factors and acoustic measures were also explored.

Results

The children with CIs showed significantly poorer performance in tone differentiability and hit rate than the NH children. For both CI and NH groups, performance on the two acoustic measures was highly correlated with each other (r values: 0.895–0.961). The performance between the two methods (i.e., 2D and 3D methods) was also highly correlated (r values: 0.774–0.914). Age at implantation and duration of CI use showed a weak correlation with the scores of acoustic measures under both methods. These two factors jointly accounted for 15.4–18.9% of the total variance of tone production performance.

Conclusion

There were significant deficits in tone production ability in most prelingually deafened children with CIs, even after prolonged use of the devices. The strong correlation between the two methods suggested that the simpler, 2D method seemed to be efficient in acoustic assessment for lexical tones in hearing-impaired children. Age at implantation and especially the duration of CI use were significant, although weak, predictors for tone development in pediatric CI users. Although a large part of tone production ability could not be attributed to these two factors, the results still encourage early implantation and continual CI use for better lexical tone development in Mandarin-speaking pediatric CI users.

Development and evaluation of methods for assessing tone production skills in Mandarin-speaking children with cochlear implants

The aim of the study was (1) to develop methods for evaluating tone production of children with cochlear implants (CIs) who speak Mandarin Chinese and (2) to evaluate the efficacy of using these methods to assess tone production. The subjects included two groups of native-Mandarin-Chinese-speaking children: 14 prelingually deafened children who had received CIs and 61 normal-hearing (NH) children as controls. The acoustic analysis focused on quantification of the degree of differentiation among lexical tones based on tonal ellipses and the overall similarity of tone contours produced by the children with CIs to normative contours derived from the 61 NH children. An artificial neural network was used to recognize tones produced by the children with CIs after trained with tone tokens produced by the NH children. Finally, perceptual judgments on the tone production of both groups were obtained from eight native-Mandarin-speaking NH adults to evaluate the efficacy of the methods. The results showed that all measures using the acoustic, neural-network, and perceptual analyses were highly correlated with each other and could be used to effectively evaluate tone production of children with CIs.

Recognition of lexical tone production of children with an artificial neural network

CONCLUSION

This study demonstrated that the artificial neural network can successfully classify Mandarin Chinese tone patterns produced by multiple children. The neural network can be used as an objective way of evaluating tone production of children.

OBJECTIVES

Traditionally, tone production is evaluated subjectively using human listeners. The aim of the present study was to investigate the efficacy of using an artificial neural network in evaluating tone production of Mandarin-speaking children.

SUBJECTS AND METHODS

Speech materials were recorded from 61 normal-hearing children. The fundamental frequency (F0) of each monosyllabic word was extracted and then used as inputs to a feed-forward backpropagation artificial neural network. The number of inputs was set at 12, whereas the number of hidden neurons was set at 16 in the neural network. The output layer consisted of four neurons representing the four Mandarin tone patterns. The tone recognition performance of the neural network was further compared with that of native Mandarin-speaking adult listeners.

RESULTS

The neural network successfully classified the tone patterns of the 61 child speakers with an accuracy of about 85% correct. This high accuracy exceeded the tone recognition performance by the adult listeners. Individual child speakers showed varied tone production accuracy as recognized by the adult listeners or by the neural network.