In this paper, modern numerical continuation methodologies are presented as a way of understanding and computing multiplicity of solutions in undergraduate physics problems. Mechanical and thermodynamical problems are used as a storyline to introduce the mathematical formalism required to clarify the distinction between the uniqueness and multiplicity of equilibrium solutions and the critical states of a nonlinear physical problem, as well as to illustrate how these novel numerical continuation techniques are implemented in practice. The paper provides simple numerical MATLAB codes that are easily adaptable to other problems, as well as updated software and literature resources.