The MAMe dataset: On the relevance of high resolution and variable shape image properties

Abstract

The mostcommon approach in image classification tasks is to resize all images in the dataset to a unique shape, while reducing their resolution to a size that makes experimentation at scale easier. This practice has benefits from a computational perspective, but it entails negative side-effects on performance due to loss of information and image deformation. In this work we introduce the MAMe dataset, an image classification dataset with remarkable properties of high resolution and variable shape. The goal of MAMe is to provide a tool for studying the impact of such properties in image classification, while motivating research in the topic. The MAMe dataset contains thousands of artworks from three different museums, and proposes a classification task consisting on differentiating between 29 mediums (i.e., materials and techniques) supervised by art experts. After analyzing the novelty of MAMe in the context of the current image classification tasks, a thorough description of the task is provided, along with statistics of the dataset. Experiments are conducted to evaluate the impact of using high resolution images, variable shape inputs, as well as both properties at the same time. Results illustrate the positive impact in performance when using high resolution images, while highlighting the lack of solutions to exploit variable shapes. An additional experiment exposes the distinctiveness between the MAMe dataset and the prototypical ImageNet dataset, showing that performance improves due to information gain and resolution gain. Finally, the baselines are inspected using explainability methods and expert knowledge, in order to gain insights about the challenges that remain ahead.

This work is partially supported by the Intel-BSC Exascale Lab agreement, by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project, by the Generalitat de Catalunya (contracts 2017-SGR-1414) and by the Secretaria d’Universitats i Recerca of the Generalitat de Catalunya under the Industrial Doctorate Grant DI 2018-100. Authors would like to thank the support and assessment of the Conservació-Restauració del Patrimoni group (2017-SGR-1151).

Peer Reviewed

Postprint (author's final draft)