How to represent a word and predict it, too: improving tied architectures for language modelling

Home | About RECERCAT | Contact

Català | Castellano

All of RECERCAT

By Communities &
Collections By Defense Date By Authors By Titles By Subject

This Collection

By Defense Date By Authors By Titles By Subject

Statistics

View Statistics All RECERCAT

My RECERCAT

Other repositories directory

RECERCAT Home > Universitat Pompeu Fabra > Articles, congressos, llibres > View document

To access the full text documents, please follow this link: http://hdl.handle.net/10230/35675

dc.contributor.author	Gulordava, Kristina
dc.contributor.author	Aina, Laura
dc.contributor.author	Boleda, Gemma
dc.date	2018
dc.identifier.citation	Gulordava K, Aina L, Boleda G. How to represent a word and predict it, too: improving tied architectures for language modelling. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg: Association for Computational Linguistics; 2018. p. 2936–41.
dc.identifier.uri	http://hdl.handle.net/10230/35675
dc.format	application/pdf
dc.language.iso	eng
dc.publisher	ACL (Association for Computational Linguistics)
dc.relation	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg: Association for Computational Linguistics; 2018. p. 2936–41.
dc.relation	info:eu-repo/grantAgreement/EC/H2020/715154
dc.rights	© ACL, Creative Commons Attribution 4.0 License
dc.rights	info:eu-repo/semantics/openAccess
dc.rights	http://creativecommons.org/licenses/by/4.0/
dc.subject	Language models
dc.subject	Word embeddings
dc.subject	Neural networks
dc.subject	Tied representations
dc.title	How to represent a word and predict it, too: improving tied architectures for language modelling
dc.type	info:eu-repo/semantics/conferenceObject
dc.type	info:eu-repo/semantics/publishedVersion
dc.description.abstract	Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018.
dc.description.abstract	Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks.
dc.description.abstract	This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).

Show simple item record

All of RECERCAT

This Collection

Statistics

My RECERCAT

Related documents

Other documents of the same author