Promoting generalized cross-lingual question answering in few-resource scenarios via self-knowledge distillation

Abstract

We address the challenge of Generalized Cross-Lingual Transfer (G-XLT) in extractive Question Answering, where question and context languages differ, a problem particularly difficult for low-resource languages. Working with only a thousand parallel QA samples, we combine cross-lingual sampling with self-knowledge distillation to regularize cross-lingual fine-tuning. We introduce the novel mean Average Precision at k (mAP@k) coefficient, which mitigates the negative impact of incorrect predictions during training and serves as a diagnostic tool providing early training guidance and reliable indicators of model learning. Evaluations on MLQA, XQuAD, and TyDiQA-GoldP datasets demonstrate that our approach consistently outperforms standard cross-entropy fine-tuning of the mBERT multilingual model. Our method represents a promising alternative to machine translation-based approaches, particularly valuable for low-resource languages where translation quality is poor, offering an efficient solution for cross-lingual transfer in data-scarce settings.

This work was supported by the project PID2019-107579RB-I00 (MICINN).

Peer Reviewed

Postprint (published version)