A vision-based obstacle detection and avoidance algorithm for Unmanned Aerial Vehicles (UAVs) in fast and low flights (‘Nap of the Earth’ flights) has been developed. First, a literature review has been performed for both obstacle detection methods and obstacle avoidance methods. Two main categories have been recognised amongst the approaches used for each of them: the use of either a monocular camera or a stereo camera for the obstacle detection; and either reactive or anticipative methods for the obstacle avoidance. The implemented algorithm uses a stereo camera and an anticipative approach, as they have been identified as the most suitable techniques for fast flights. The stereo pairs of images are used to construct a 3D global occupancy map with a novel cell tagging method. This map is then used to calculate a set of waypoints which can lead the vehicle safely towards its goal. Finally, a novel optimisation algorithm is used to compute the fastest feasible polynomial trajectory along this set of waypoints to bring the vehicle to the target in the minimum achievable amount of time.