Model-based synthesis of plucked string instruments by using a class of scattering recurrent networks

A Novel Prosodic-Information Synthesizer Based on Recurrent Fuzzy Neural Network for the Chinese TTS System

In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.

A class of physical modeling recurrent networks for analysis/synthesis of plucked string instruments

A new approach is proposed that closely synthesizes tones of plucked string instruments by using a class of physical modeling recurrent networks. The strategies employed consist of a fast training algorithm and a multistage training procedure that are able to obtain the synthesis parameters for a specific instrument automatically. The training vector can be recorded tones of most target plucked instruments with ordinary microphones. The proposed approach delivers encouraging results when it is applied to different types of plucked string instruments such as steel-string guitar, nylon-string guitar, harp, Chin, Yueh-chin, and Pipa. The synthesized tones sound very close to the originals produced by their acoustic counterparts. In addition, the paper presents an embedded technique that can produce special effects such as vibrato and portamento that are vital to the playing of plucked-string instruments. The computation required in the resynthesis processing is also reasonable.