Long gap between request's response times (shows with ssl indicatin color)

Hi

In my test there are 40 requests>after 5th request the timeline shows SSL (purple) color for almost 4seconds (no request in this time shows) before 6th request. Not sure is it due to TLS handshake which needs to be improved or how to analyze this gap. Kindly guide.