Most likely the browser is busy doing something else - could be loading js, parsing huge style sheets, etc. In some cases with some test environments the browser is actually updating itself or plugins.

Take a look at the CPU Utilization graph that appears underneath your waterfall - is that high?