Motion from sound: Intermodal neural network mapping

Tetsuya Ogata, Hiroshi G. Okuno, Hideki Kozima

Research output: Contribution to journalArticlepeer-review


A technological method has been developed for intermodal mapping to generate robot motion from various sounds as well as to generate sounds from motions. The procedure consists of two phases, first the learning phase in which it observes some events together with associated sounds and then memorizes those sounds along with the motions of the sound source. Second phase is the interacting phase in which the robot receives limited sensory information from a single modality as input and associates this with different modality and expresses it. The recurrent-neural-network model with parametric bias (RNNPB) is applied that uses the current state-vector as input for outputting the next state-vector. The RNNPB model can self-organize the values that encode the input dynamics into special parametric-bias modes to reproduce he multimodal sensory flow.

Original languageEnglish
Article number4475863
Pages (from-to)76-78
Number of pages3
JournalIEEE Intelligent Systems
Issue number2
Publication statusPublished - 2008 Mar


Dive into the research topics of 'Motion from sound: Intermodal neural network mapping'. Together they form a unique fingerprint.

Cite this