Training a language model using webdata for large vocabulary Japanese spontaneous speech recognition

Ryo Masumura; Seongjun Hahm; Akinori Ito

Training a language model using webdata for large vocabulary Japanese spontaneous speech recognition

Ryo Masumura, Seongjun Hahm, Akinori Ito

ENG - Communications Engineering

Tohoku University

Research output: Contribution to journal › Conference article › peer-review

12 Citations (Scopus)

Abstract

This paper describes a language modeling method using large-scale spoken language data retrieved from the Web for spontaneous speech recognition. We downloaded 15 million Web pages on a comprehensive range topics. Next, spoken language- like texts were selected from the downloaded Web data using the naïve Bayes classifier, and typical linguistic phenomena such as fillers and pauses were added using simulation models. A language model trained by the generated data gave as high performance as the large-scale spontaneous speech corpus (Corpus of Spontaneous Japanese, CSJ). By combining the generated data and CSJ, we improved word accuracy.

Original language	English
Pages (from-to)	1465-1468
Number of pages	4
Journal	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Publication status	Published - 2011
Event	12th Annual Conference of the International Speech Communication Association, INTERSPEECH 2011 - Florence, Italy Duration: 2011 Aug 27 → 2011 Aug 31

Keywords

Corpus of Spontaneous Japanese
Language model
Large vocabulary continuous speech recognition
Spontaneous speech recognition
World Wide Web

Cite this

@article{c4ec615beaa24489967871e53e00b65e,

title = "Training a language model using webdata for large vocabulary Japanese spontaneous speech recognition",

abstract = "This paper describes a language modeling method using large-scale spoken language data retrieved from the Web for spontaneous speech recognition. We downloaded 15 million Web pages on a comprehensive range topics. Next, spoken language- like texts were selected from the downloaded Web data using the na{\"i}ve Bayes classifier, and typical linguistic phenomena such as fillers and pauses were added using simulation models. A language model trained by the generated data gave as high performance as the large-scale spontaneous speech corpus (Corpus of Spontaneous Japanese, CSJ). By combining the generated data and CSJ, we improved word accuracy.",

keywords = "Corpus of Spontaneous Japanese, Language model, Large vocabulary continuous speech recognition, Spontaneous speech recognition, World Wide Web",

author = "Ryo Masumura and Seongjun Hahm and Akinori Ito",

year = "2011",

language = "English",

pages = "1465--1468",

journal = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

issn = "1990-9772",

note = "12th Annual Conference of the International Speech Communication Association, INTERSPEECH 2011 ; Conference date: 27-08-2011 Through 31-08-2011",

}

TY - JOUR

T1 - Training a language model using webdata for large vocabulary Japanese spontaneous speech recognition

AU - Masumura, Ryo

AU - Hahm, Seongjun

AU - Ito, Akinori

PY - 2011

Y1 - 2011

N2 - This paper describes a language modeling method using large-scale spoken language data retrieved from the Web for spontaneous speech recognition. We downloaded 15 million Web pages on a comprehensive range topics. Next, spoken language- like texts were selected from the downloaded Web data using the naïve Bayes classifier, and typical linguistic phenomena such as fillers and pauses were added using simulation models. A language model trained by the generated data gave as high performance as the large-scale spontaneous speech corpus (Corpus of Spontaneous Japanese, CSJ). By combining the generated data and CSJ, we improved word accuracy.

AB - This paper describes a language modeling method using large-scale spoken language data retrieved from the Web for spontaneous speech recognition. We downloaded 15 million Web pages on a comprehensive range topics. Next, spoken language- like texts were selected from the downloaded Web data using the naïve Bayes classifier, and typical linguistic phenomena such as fillers and pauses were added using simulation models. A language model trained by the generated data gave as high performance as the large-scale spontaneous speech corpus (Corpus of Spontaneous Japanese, CSJ). By combining the generated data and CSJ, we improved word accuracy.

KW - Corpus of Spontaneous Japanese

KW - Language model

KW - Large vocabulary continuous speech recognition

KW - Spontaneous speech recognition

KW - World Wide Web

UR - http://www.scopus.com/inward/record.url?scp=84865740959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865740959&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84865740959

SN - 1990-9772

SP - 1465

EP - 1468

JO - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

JF - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

T2 - 12th Annual Conference of the International Speech Communication Association, INTERSPEECH 2011

Y2 - 27 August 2011 through 31 August 2011

ER -

Training a language model using webdata for large vocabulary Japanese spontaneous speech recognition

Abstract

Keywords

Other files and links

Fingerprint

Cite this