However you step into the wonderful world of deep learning, the learning rate will always appear on your journey. This article distills a series of previous works and describes why we need a learning rate, how to choose one, it's role during training, and how to handle it when using pretrained weights.