Scene text detection and tracking for wearable text-to-speech translation camera

Hideaki Goto; Kunqi Liu

doi:10.1007/978-3-319-41267-2_4

Scene text detection and tracking for wearable text-to-speech translation camera

Hideaki Goto, Kunqi Liu

Research and Development Divisions

Tohoku University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Camera-based character recognition applications equipped with voice synthesizer are useful for the blind to read text messages in the environments. Such applications in the current market and/or similar prototypes under research require users’ active reading actions, which hamper other activities. We presented a different approach at ICCHP2014; the user can be passive, while the device actively finds useful text in the scene. Text tracking feature was introduced to avoid duplicate reading of the same text. This report presents an improved system with two key components, scene text detection and tracking, that can handle text in various languages including Japanese/Chinese and resolve some scene analysis problems such as merging of text lines. We have employed the MSER (Maximally Stable Extremal Regions) algorithm to obtain better text images, and developed a new text validation filter. Some technical challenges for future device design are presented as well.

Original language	English
Title of host publication	Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings
Editors	Christian Bühler, Petr Penaz, Klaus Miesenberger
Publisher	Springer Verlag
Pages	23-26
Number of pages	4
ISBN (Print)	9783319412665
DOIs	https://doi.org/10.1007/978-3-319-41267-2_4
Publication status	Published - 2016
Event	15th International Conference on Computers Helping People with Special Needs, ICCHP 2016 - Linz, Austria Duration: 2016 Jul 13 → 2016 Jul 15

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9759
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th International Conference on Computers Helping People with Special Needs, ICCHP 2016
Country/Territory	Austria
City	Linz
Period	16/7/13 → 16/7/15

Keywords

Reading assistant
Scene text recognition
Text tracking
Text-to-speech
Wearable camera

Access to Document

10.1007/978-3-319-41267-2_4

Cite this

Goto, H., & Liu, K. (2016). Scene text detection and tracking for wearable text-to-speech translation camera. In C. Bühler, P. Penaz, & K. Miesenberger (Eds.), Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings (pp. 23-26). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9759). Springer Verlag. https://doi.org/10.1007/978-3-319-41267-2_4

Goto, Hideaki ; Liu, Kunqi. / Scene text detection and tracking for wearable text-to-speech translation camera. Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings. editor / Christian Bühler ; Petr Penaz ; Klaus Miesenberger. Springer Verlag, 2016. pp. 23-26 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{78f3d05bd6dc44628c475d00958c136c,

title = "Scene text detection and tracking for wearable text-to-speech translation camera",

abstract = "Camera-based character recognition applications equipped with voice synthesizer are useful for the blind to read text messages in the environments. Such applications in the current market and/or similar prototypes under research require users{\textquoteright} active reading actions, which hamper other activities. We presented a different approach at ICCHP2014; the user can be passive, while the device actively finds useful text in the scene. Text tracking feature was introduced to avoid duplicate reading of the same text. This report presents an improved system with two key components, scene text detection and tracking, that can handle text in various languages including Japanese/Chinese and resolve some scene analysis problems such as merging of text lines. We have employed the MSER (Maximally Stable Extremal Regions) algorithm to obtain better text images, and developed a new text validation filter. Some technical challenges for future device design are presented as well.",

keywords = "Reading assistant, Scene text recognition, Text tracking, Text-to-speech, Wearable camera",

author = "Hideaki Goto and Kunqi Liu",

note = "Funding Information: A part of this work was supported by the Grant-in-Aid 14–15 in 2014 from the Okawa Foundation in Japan. Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 15th International Conference on Computers Helping People with Special Needs, ICCHP 2016 ; Conference date: 13-07-2016 Through 15-07-2016",

year = "2016",

doi = "10.1007/978-3-319-41267-2_4",

language = "English",

isbn = "9783319412665",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "23--26",

editor = "Christian B{\"u}hler and Petr Penaz and Klaus Miesenberger",

booktitle = "Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings",

address = "Germany",

}

Goto, H & Liu, K 2016, Scene text detection and tracking for wearable text-to-speech translation camera. in C Bühler, P Penaz & K Miesenberger (eds), Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9759, Springer Verlag, pp. 23-26, 15th International Conference on Computers Helping People with Special Needs, ICCHP 2016, Linz, Austria, 16/7/13. https://doi.org/10.1007/978-3-319-41267-2_4

Scene text detection and tracking for wearable text-to-speech translation camera. / Goto, Hideaki; Liu, Kunqi.
Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings. ed. / Christian Bühler; Petr Penaz; Klaus Miesenberger. Springer Verlag, 2016. p. 23-26 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9759).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Scene text detection and tracking for wearable text-to-speech translation camera

AU - Goto, Hideaki

AU - Liu, Kunqi

N1 - Funding Information: A part of this work was supported by the Grant-in-Aid 14–15 in 2014 from the Okawa Foundation in Japan. Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - Camera-based character recognition applications equipped with voice synthesizer are useful for the blind to read text messages in the environments. Such applications in the current market and/or similar prototypes under research require users’ active reading actions, which hamper other activities. We presented a different approach at ICCHP2014; the user can be passive, while the device actively finds useful text in the scene. Text tracking feature was introduced to avoid duplicate reading of the same text. This report presents an improved system with two key components, scene text detection and tracking, that can handle text in various languages including Japanese/Chinese and resolve some scene analysis problems such as merging of text lines. We have employed the MSER (Maximally Stable Extremal Regions) algorithm to obtain better text images, and developed a new text validation filter. Some technical challenges for future device design are presented as well.

AB - Camera-based character recognition applications equipped with voice synthesizer are useful for the blind to read text messages in the environments. Such applications in the current market and/or similar prototypes under research require users’ active reading actions, which hamper other activities. We presented a different approach at ICCHP2014; the user can be passive, while the device actively finds useful text in the scene. Text tracking feature was introduced to avoid duplicate reading of the same text. This report presents an improved system with two key components, scene text detection and tracking, that can handle text in various languages including Japanese/Chinese and resolve some scene analysis problems such as merging of text lines. We have employed the MSER (Maximally Stable Extremal Regions) algorithm to obtain better text images, and developed a new text validation filter. Some technical challenges for future device design are presented as well.

KW - Reading assistant

KW - Scene text recognition

KW - Text tracking

KW - Text-to-speech

KW - Wearable camera

UR - http://www.scopus.com/inward/record.url?scp=84978882431&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978882431&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-41267-2_4

DO - 10.1007/978-3-319-41267-2_4

M3 - Conference contribution

AN - SCOPUS:84978882431

SN - 9783319412665

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 23

EP - 26

BT - Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings

A2 - Bühler, Christian

A2 - Penaz, Petr

A2 - Miesenberger, Klaus

PB - Springer Verlag

T2 - 15th International Conference on Computers Helping People with Special Needs, ICCHP 2016

Y2 - 13 July 2016 through 15 July 2016

ER -

Goto H, Liu K. Scene text detection and tracking for wearable text-to-speech translation camera. In Bühler C, Penaz P, Miesenberger K, editors, Computers Helping People with Special Needs - 15th International Conference, ICCHP 2016, Proceedings. Springer Verlag. 2016. p. 23-26. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-41267-2_4

Scene text detection and tracking for wearable text-to-speech translation camera

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this