There is increased awareness of the possibility of developmental memories resulting from evolutionary learning. Genetic regulatory and neural networks can be modelled by analogous formalism raising the important question of productive analogies in principles, processes and performance. We investigate the formation and persistence of various developmental memories of past phenotypes asking how the number of remembered past phenotypes scales with network size, to what extent memories stored form by Hebbian-like rules, and how robust these developmental "devo-engrams" are against networks perturbations (graceful degradation). The analogy between neural and genetic regulatory networks is not superficial in that it allows knowledge transfer between fields that used to be developed separately from each other. Known examples of spectacular phenotypic radiations could partly be accounted for in such terms. The development of individual organisms from embryo to adult state is under the control of many genes. During development the initially active genes activate other genes, which in turn change the composition of regulatory elements. The behavior of genetic regulatory systems shows similarities to that of neural networks, of which the most remarkable one is developmental memory, the ability to quickly adapt to environments that have occurred in the past, occasionally several generations earlier. This is because each previously evolved developmental pathway leaves an "imprint" in the gene regulatory network. We investigated the properties of this system; the number of different developmental pathways that can be "memorized", how this number depends on the number of expressed genes, how fast the system can switch between these pathways, and its robustness against various disturbances affecting either the embryo state or the gene interaction networks. Our results suggest that developmental memory may also provide the mechanism behind some rapid speciation processes.