When is the right time to ask for a second date?

I met this guy on Tinder and we went out to a movie. Obviously there was something between us because he did the hold hand, put arm around me, and stroke my leg. He left over the weekend and didn't want to bother him till he came back to town. How soon after the first date should there be a second date?