Meta-UNet: enhancing skin-lesion segmentation with multimodal feature integration and uncertainty estimation

Abstract

Purpose. Medical image segmentation plays a crucial role in diagnostic pipelines. This study investigates the integration of lesion-specific metadata with image data to enhance segmentation accuracy and reduce predictive uncertainty. Methods. The standard U-Net architecture was modified to incorporate lesion-specific metadata (Meta-UNet). Various integration strategies, including addition, weighted addition, and embedding layers, were evaluated. Additionally, a Bayesian Meta-UNet with Monte Carlo Dropout (MCD) was developed to assess the impact of metadata integration on model uncertainty. Uncertainty was quantified using measures such as Confidence Maps, Entropy, Mutual Information, and Expected Pairwise Kullback'Leibler divergence (EPKL). An aggregation strategy was also introduced to provide a single comprehensive uncertainty score per image. Results. Meta-UNet outperformed standard U-Net across PH2, ISIC 2018, and HAM10000 datasets. On PH2, it achieved 84.64% accuracy and 90.62% Intersection over Union (IoU), compared to 83.36% and 89.19%. On ISIC 2018, U-Net scored 71.02 ± 6.69 IoU and 79.89 ± 5.09 Dice. On HAM10000, Meta-UNet achieved 88.66 ± 6.09 IoU and 93.42 ± 5.19 Dice. Meta-UNet reduced uncertainty (e.g., 0.149 vs. 0.1745), highlighting the benefit of metadata integration in improving segmentation accuracy and model confidence. Conclusion. Integrating lesion-specific metadata into the U-Net architecture significantly improves segmentation accuracy and reduces predictive uncertainty. The inclusion of metadata enhances model confidence and reliability, underscoring its potential to strengthen diagnostic segmentation pipelines.