And here is a link to Ye’s paper giving a strongly polynomial algorithm for Markov decision processes with constant discounting. Update: The slides for Ye’s lecture are now posted and they are very detailed and clear.