01697429

Component

Safe and efficient autonomous driving maneuvers in an interactive and complex environment can be considerably challenging due to the unpredictable actions of other surrounding agents that may be cooperative or adversarial in their interactions with the ego vehicle. One of the state-of-the-art approaches is to apply Reinforcement Learning (RL) to learn a time-sequential driving policy, to execute proper control strategy or tracking trajectory in dynamic situations. However, direct application of RL algorithms is not satisfactorily enough to deal with the cases in the autonomous driving domain, mainly due to the complex driving environment and continuous action space. In this paper, the authors adopt Q-learning as their basic learning framework and design a unique format of the Q-function approximator that consists of neural networks to handle the continuous action space challenge. The learning model is present in a closed form of continuous control variables and trained in a simulation platform that the authors have developed with embedded properties of real-time vehicle interactions. The proposed algorithm avoids invoking an additional ‘actor’ network that learns to take actions, as in actor-critic algorithms. At the same time, some prior knowledge of vehicle dynamics is also fed into the model to assist learning. The authors test their algorithm with a challenging use case - lane change maneuver, to verify the practicability and feasibility of the proposed approach. Results from accumulated rewards and vehicle performance show that RL vehicle agents successfully learn a safe, comfort and efficient driving policy as defined in the reward function.

This paper was sponsored by TRB committee AHB30 Standing Committee on Vehicle-Highway Automation. Alternate title: Driving Maneuver Learning via Reinforcement Learning for Automated Vehicles in An Interactive Environment

19-01116

English

5p

2019

Transportation Research Board 98th Annual Meeting

Digital/other

Figures; References

Advanced vehicle control systems; Autonomous vehicles; Intelligent agents; Lane changing; Machine learning; Neural networks

Deep learning; Reinforcement learning

Data and Information Technology; Highways; Vehicles and Equipment

Transportation Research Board Annual Meeting 2019 Paper #19-01116

TRIS, TRB, ATRI

Dec 7 2018 9:27AM