TY - GEN
T1 - Synthesis of photo-realistic facial animation from text based on HMM and DNN with animation unit
AU - Sato, Kazuki
AU - Nose, Takashi
AU - Ito, Akinori
N1 - Funding Information:
Part of this work was supported by JSPS KAKENHI Grant Number JP15H02720.
Publisher Copyright:
© Springer International Publishing AG 2017.
PY - 2017
Y1 - 2017
N2 - In this paper, we propose a technique for synthesizing photorealistic facial animation from a text based on hidden Markov model (HMM) and deep neural network (DNN) with facial features for an interactive agent implementation. In the proposed technique, we use Animation Unit (AU) as facial features that express the state of each part of face and can be obtained by Kinect. We synthesize facial features from any text using the same framework as the HMM-based speech synthesis. Facial features are generated from HMM and are converted into intensities of pixels using DNN. We investigate appropriate conditions for training of HMM and DNN. Then, we perform an objective evaluation to compare the proposed technique with a conventional technique based on the principal component analysis (PCA).
AB - In this paper, we propose a technique for synthesizing photorealistic facial animation from a text based on hidden Markov model (HMM) and deep neural network (DNN) with facial features for an interactive agent implementation. In the proposed technique, we use Animation Unit (AU) as facial features that express the state of each part of face and can be obtained by Kinect. We synthesize facial features from any text using the same framework as the HMM-based speech synthesis. Facial features are generated from HMM and are converted into intensities of pixels using DNN. We investigate appropriate conditions for training of HMM and DNN. Then, we perform an objective evaluation to compare the proposed technique with a conventional technique based on the principal component analysis (PCA).
KW - Animation unit
KW - Deep neural network
KW - Face image synthesis
KW - Hidden Markov model
KW - Photo-realistic facial animation
UR - http://www.scopus.com/inward/record.url?scp=85006010347&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85006010347&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-50212-0_4
DO - 10.1007/978-3-319-50212-0_4
M3 - Conference contribution
AN - SCOPUS:85006010347
SN - 9783319502113
T3 - Smart Innovation, Systems and Technologies
SP - 29
EP - 36
BT - Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2016
A2 - Pan, Jeng-Shyang
A2 - Tsai, Pei-Wei
A2 - Huang, Hsiang-Cheh
PB - Springer Science and Business Media Deutschland GmbH
T2 - 12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2016
Y2 - 21 November 2016 through 23 November 2016
ER -