Right-truncatable neural word embeddings

Jun Suzuki; Masaaki Nagata

doi:10.18653/v1/n16-1135

Right-truncatable neural word embeddings

Jun Suzuki, Masaaki Nagata

Center for Data-driven Science and Artificial Intelligence (CDS)

Nippon Telegraph & Telephone

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

Abstract

This paper proposes an incremental learning strategy for neural word embedding methods, such as SkipGrams and Global Vectors. Since our method iteratively generates embedding vectors one dimension at a time, obtained vectors equip a unique property. Namely, any right-truncated vector matches the solution of the corresponding lower-dimensional embedding. Therefore, a single embedding vector can manage a wide range of dimensional requirements imposed by many different uses and applications.

Original language	English
Title of host publication	2016 Conference of the North American Chapter of the Association for Computational Linguistics
Subtitle of host publication	Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference
Publisher	Association for Computational Linguistics (ACL)
Pages	1145-1151
Number of pages	7
ISBN (Electronic)	9781941643914
DOIs	https://doi.org/10.18653/v1/n16-1135
Publication status	Published - 2016
Event	15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - San Diego, United States Duration: 2016 Jun 12 → 2016 Jun 17

Publication series

Name	2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference

Conference

Conference	15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016
Country/Territory	United States
City	San Diego
Period	16/6/12 → 16/6/17

Access to Document

10.18653/v1/n16-1135

Cite this

Suzuki, J., & Nagata, M. (2016). Right-truncatable neural word embeddings. In 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference (pp. 1145-1151). (2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/n16-1135

Suzuki, Jun ; Nagata, Masaaki. / Right-truncatable neural word embeddings. 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference. Association for Computational Linguistics (ACL), 2016. pp. 1145-1151 (2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference).

@inproceedings{8ae84eed47a74e159ec8c5a466772c08,

title = "Right-truncatable neural word embeddings",

abstract = "This paper proposes an incremental learning strategy for neural word embedding methods, such as SkipGrams and Global Vectors. Since our method iteratively generates embedding vectors one dimension at a time, obtained vectors equip a unique property. Namely, any right-truncated vector matches the solution of the corresponding lower-dimensional embedding. Therefore, a single embedding vector can manage a wide range of dimensional requirements imposed by many different uses and applications.",

author = "Jun Suzuki and Masaaki Nagata",

note = "Publisher Copyright: {\textcopyright}2016 Association for Computational Linguistics.; 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 ; Conference date: 12-06-2016 Through 17-06-2016",

year = "2016",

doi = "10.18653/v1/n16-1135",

language = "English",

series = "2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "1145--1151",

booktitle = "2016 Conference of the North American Chapter of the Association for Computational Linguistics",

address = "United States",

}

Suzuki, J & Nagata, M 2016, Right-truncatable neural word embeddings. in 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference. 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference, Association for Computational Linguistics (ACL), pp. 1145-1151, 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016, San Diego, United States, 16/6/12. https://doi.org/10.18653/v1/n16-1135

Right-truncatable neural word embeddings. / Suzuki, Jun; Nagata, Masaaki.
2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference. Association for Computational Linguistics (ACL), 2016. p. 1145-1151 (2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Right-truncatable neural word embeddings

AU - Suzuki, Jun

AU - Nagata, Masaaki

PY - 2016

Y1 - 2016

N2 - This paper proposes an incremental learning strategy for neural word embedding methods, such as SkipGrams and Global Vectors. Since our method iteratively generates embedding vectors one dimension at a time, obtained vectors equip a unique property. Namely, any right-truncated vector matches the solution of the corresponding lower-dimensional embedding. Therefore, a single embedding vector can manage a wide range of dimensional requirements imposed by many different uses and applications.

AB - This paper proposes an incremental learning strategy for neural word embedding methods, such as SkipGrams and Global Vectors. Since our method iteratively generates embedding vectors one dimension at a time, obtained vectors equip a unique property. Namely, any right-truncated vector matches the solution of the corresponding lower-dimensional embedding. Therefore, a single embedding vector can manage a wide range of dimensional requirements imposed by many different uses and applications.

UR - http://www.scopus.com/inward/record.url?scp=84994193602&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994193602&partnerID=8YFLogxK

U2 - 10.18653/v1/n16-1135

DO - 10.18653/v1/n16-1135

M3 - Conference contribution

AN - SCOPUS:84994193602

T3 - 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference

SP - 1145

EP - 1151

BT - 2016 Conference of the North American Chapter of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016

Y2 - 12 June 2016 through 17 June 2016

ER -

Suzuki J, Nagata M. Right-truncatable neural word embeddings. In 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference. Association for Computational Linguistics (ACL). 2016. p. 1145-1151. (2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference). doi: 10.18653/v1/n16-1135

Right-truncatable neural word embeddings

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this