I did many improvements in latest tip of rhodecode and vcs, also I made some tests on artificial repos containing a lot of binary files, i things look much better now. Can You test Your use case on the problematic repository ?