Page loading problem very slow response time

I have Nginx server on my VPS, and since a while I noticed a heavy page load on most pages. I have vbuletin board installed on my server working just fine without any issues expect the pages load very slow.

I have tested my website many times using different website speed testing tools, and kind of I get the same results. The main issue is mostly server response time as observed from testing tools results.

for example, when I tested my site on google insight, I get many issues such as

Eliminate render-blocking JavaScript and CSS in above-the-fold content
I have only 2 css files on my main website page and 4 js scripts, which in overall shouldn't be a big deal on my server.

I have to mention that my website's static content on subdomain is served by CDN Cloudflare and the main website html pages are compressed by Cloudflare only.

other website testing tools suggest the same issue, which is server response time as the main problem. yet I don't know really how to deal with this issue and where to start since I've checked my server many times and got all no issues with it.

This is another result shows that First Bye Time is 1.230s, which is abnormal.