Deep Reinforcement Learning for Sustainable and Energy-Efficient Autonomous Navigation in Real-World Environments

Abstract

Growing concerns over climate change and ecological degradation have brought sustainability to the forefront of the global agenda. In parallel, the urgency for developing electric mobility technologies has heightened to combat air pollution in urban areas. Reinforcement learning (RL) can provide solutions for optimal, risk-sensitive, and energy-efficient motion planning of autonomous vehicles in unknown and dynamic environments, but typically does not consider energy consumption as an explicit objective during training. Moreover, several real-world constraints related to safety and comfort are frequently neglected. In this work, sustainability-focused deep RL algorithms are proposed to enable energy-efficient autonomous vehicle navigation under real-world constraints. The performance of different approaches is assessed in a simulated two-dimensional environment, and their ability to reduce energy consumption while still satisfying real-world constraints is evaluated

Diverse approaches are explored to optimize energy-aware RL policy learning, including the combination of model-based and model-free techniques, the design of reward structures that promote energy-efficient behaviours, the application of safety layers to guarantee constraint satisfaction, and the consideration of real-world variations in the environment. The systematic exploitation of these approaches is then employed to tackle three specific challenges: (i) the development of energy-aware models for planning and policy learning; (ii) the exploitation of models in a safely constrained exploration of the state-action space; and (iii) the balancing of safety and energy efficiency during policy training