We predict the existence of lateral drag forces near the flat surface of an absorbing slab made of an anisotropic material. The forces originate from the fluctuations of the electromagnetic field, when the anisotropy axis of the material forms a certain angle with the surface. In this situation, the spatial spectra of the fluctuating electromagnetic fields becomes asymmetric, different for positive and negative transverse wave vectors components. Differently from the case of van der Waals interactions in which the forward-backward symmetry is broken due to the particle movement, in our case the lateral motion results merely from the anisotropy of the slab. This new effect, of particular significance in hyperbolic materials, could be used for the manipulation of nanoparticles.