These few days i had try to do the research about this TTFB, but i still not able to find out what is actually request during this TTFB and cost it so slow to get the 1st byte. Can somebody tell me what will cause this TTFB take so long?

We're not going to be able to tell much from the outside - it is going to be something in the server code or configuration. Since it is a VPS you should be able to install something like New Relic which will automatically instrument the application and tell you what part of the code is slow (or if it is because of a back-end dependency).