Libra (me) + Virgo (him) Whirlwind

I'm looking for some advice with a whirlwind, whiplashy romance I've had with a Virgo man.

We got extremely close and comfortable very quickly into dating, connecting on all levels. All signs pointed toward us graduating into a committed relationship in the future, but neither of us wanted to rush into anything, despite not being able to keep our hands off of each other.

His Big Ex was a partner of nine years, whom he had only been separated from for about six months, and still seemed to be processing that.

During our first week, he was accepted into a school program that he warned me would take a lot of his time, on top of him already having two jobs. He stressed that this worried him, but we both said we wanted to try to make things work together.

Two weeks into dating, he slammed on the brakes hard after a night away we took together. He withdrew and said he didn't know what he wanted, but that he wanted us to slow down. Said that he felt guilty, which I took as a measure that he might still be attached to his former partner and is having trouble reconciling spending time with a potential new one. This mostly took the form of us not being intimate, which I was fine with in him trying to find where his comfort was.

I acquiesced his wishes for another two weeks - didn't push him, did my best to give him the space he wanted. We had lots of talks about where we were going and what the other person wanted, seemed to communicate very well and I hoped we were moving forward.

One day, when my roommate was out of town, he offered to come keep me company. We ended up engaging physically for the first time since he slammed the brakes on two weeks prior - all totally initiated by him, 100%, and was a surprise to me because I was under the impression he wanted to keep things simple. It was nice, though. We had a lot of fun playing video games and snacking between spending time in bed. He left and I felt refreshed and wonderful, like we were making progress.

The next time I saw him, he ended things with me, in a manner of speaking. He said that we couldn't see each other right now because he was worried about his school program and that he doesn't want to destroy a potential relationship by being a terrible partner through the beginning of it when he knows he would withdraw and be cold. He said a number of times that he "felt guilty" whenever we were together, by which I took to mean about his ex, but now am wondering if it meant because we were getting closer and he knew that he wouldn't be able to move forward with anything with me.

He has suffered massively from depression, and between having two jobs and this school program, he says that he can't right now. That he likes me a lot, though, and that he would like to try in the future, and will see if I am still interested when he is healthier, depression-wise, aside from the issue of him having no time with his schooling and jobs.

MY QUESTION IS - what the heck do I do about this? I started falling for this person, and it's been three weeks since we've had any contact per his request, but all I do is think about him all day and all night.

I am wondering if, as a Virgo man, he might actually have intentions to reach out to me once his life is stable, or if it was maybe just a polite way to get me off the plate. He says he will contact me again to see if I am interested when he is better - should I believe him? Or should I let him go? I feel so hurt and confused. I've never had someone shut down the potential of a relationship with me because it was going TOO well. This one came in like a wrecking ball and I have had a hard time dealing with the aftermath of it all.