End-to-End Deep Learning for Autonomous Driving in AirSim

AirSim is an open source simulator for drones and cars developed by Microsoft.
In this article, we will introduce the tutorial “Autonomous Driving using End-to-End Deep Learning: an AirSim tutorial” using AirSim.

The tutorial aims for autonomous driving like the video (15 and 90 seconds version) below.

AirSim

AirSim is an open source simulator for drones and cars developed by Microsoft AI and Research Group.
Like the previous article’s CARLA, it is based on Unreal Engine 4.
For details of AirSim, please visit the following site.

another is “Autonomous Driving using End-to-End Deep Learning: an AirSim tutorial” introduced in this article.

Autonomous Driving using End-to-End Deep Learning

Autonomous driving using end-to-end deep learning is a tutorial to estimate the steering angle from the front camera image using end-to-end deep learning. In the tutorial, it is called “hello world” of autonomous driving.
It is a style to execute the following three notebooks on Jupyter Notebook.

DataExplorationAndPreparation

DataExplorationAndPreparation visualizes the dataset in the data_raw directory and generates h5 format train/eval/test dataset in the data_cooked directory.

TrainModel

TrainModel trains and evaluates a model that estimates the steering angle from the front camera image using the train/eval dataset of the data_cooked directory.
Since the number of training epochs is set to 500, the expected training time is displayed as about 10 hours, but because early stopping is also set, training ends on 30-40 epochs (about 45 minutes).

TestModel

TestModel inputs the front camera image and state (speed) from AirSim launched in the landscape environment and outputs car_controls including steering angle to AirSim.
To launch AirSim, the following PowerShell command runs in the directory which AirSim simulator was extracted.

.\AD_Cookbook_Start_AirSim.ps1 landscape

If you want to launch in a small window (e.g. 640 x 480), the following command can be used.