Abstract:
|
Drones are rapidly increasing their activity in the
airspace worldwide. This expected growth of the number of
drones makes human-based traffic management prohibitive.
Avionics systems able to sense-and-avoid obstacles and specially
visual flight rules (VFR) traffic are under research. Moreover,
to overcome loss-link contingencies, drones have to be able to
act autonomously. In this paper we present a drone concept
with a full level of autonomy based on Deep Reinforcement
Learning (DRL). From the first flight until the accomplishment
of its final mission, the drone has no need for a pilot. The only
human intervention is the engineer programming the artificial
intelligence algorithm used to train and then to control the
drone. In this paper we present the preliminary results for an
environment which is a realistic flight simulator, and an agent
that is a quad-copter drone able to execute 3 actions. The inputs
of the agent are the current state and the accumulated reward.
Experiments include self-learning periods up to 3 days, followed
by one hundred full-autonomous flight tests. Three different DRL
algorithms were used to obtain the training models, based in
Q-learning reinforcement learning. Results are very promising,
with around an 80 percent of test flights reaching the target.
In comparison with the results of a human pilot, acting in the
same simulated environment and using the same three actions,
the DRL methods demonstrated unequal results, depending on
the learning algorithm used. We applied some enhancements in
the training, with the creation of checkpoints of the training
model every time a better solution is found. In a near future we
expect to achieve results similar to the performance of a human
pilot to support the idea of full-autonomous drones through DRL
methods |