We introduce the concept of geometric frustration in plasmonic arrays of nanoelements. In particular, we present the case of a hexagonal lattice of Au nanoasterisks arranged so that the gaps between neighboring elements are small and lead to a strong near-field dipolar coupling. Besides, far-field interactions yield higher-order collective modes around the visible region that follow the translational symmetry of the lattice. However, dipolar excitations of the gaps in the hexagonal array are geometrically frustrated for interactions beyond nearest neighbors, yielding the destabilization of the low energy modes in the near infrared. This in turn results in a slow dynamics of the optical response and a complex interplay between localized and collective modes, a behavior that shares features with geometrically frustrated magnetic systems.