I had the same problem after moving my VDI directory to an external USB 3.0 hard drive with ext4 fs.
Actually, I've noticed low speed with my virtual machines. Do you think it would be better to use XFS instead?