Comunicació presentada a IWCS 2017, celebrat a Montpellier (França) del 19 al 22 de setembre de 2017.

Reference is the crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. We propose a fully data-driven, end-toend trainable model that, while operating on continuous multimodal representations, learns to organize them into a discrete-like entity library. We also introduce a referential task to test it, cross-modal tracking. Our model beats standard neural network architectures, but is outperformed by some parametrizations of Memory Networks, another model with external memory.

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE); Marie Sklodowska-Curie grant agreement No 655577 (LOVe); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).