As the cache fills up, the CMS slows down. Why?

Browsing a site while logged in, the time taken to add the SilverStripeNavigator to the bottom of pages increases as the cache (i.e. silverstripe-cache/cache) fills up. And in the CMS, clicking between different pages also slows down by a similar amount. With an empty cache, it's fine, but with 800 files in the cache, it takes 10 seconds.

Why does the number of files in the cache slow down the CMS so much? Is it opening & reading all the files?

See the SS profiler stats below. All tests are for the home page, but with more and more files in the cache. Apparently the profiler is not that accurate, but you can see the trend. SilverStripeNavigator takes longer with more files in the cache, while everything else takes roughtly the same amount of time.

The reason there are so many files in the cache is that I'm using partial cacheing, with nested cache statements. But even if there was one cache statement per page, you could easily end up with hundreds of cached files.

Putting a SS_Backtrace::backtrace() in there shows it is called by SilverStripeNavigatorItem_ArchiveLink->getHTML i.e. every time the navigator is shown.

It's a massive stack, but the crucial bit is...

Line 429 of Core.php
Zend_Cache_Core->clean(matchingAnyTag,Array)
Line 53 of Aggregate.php
Aggregate::flushCache(SiteTree)
Line 2828 of DataObject.php
DataObject->flushCache()
Line 763 of Versioned.php
Versioned::get_one_by_stage(SiteTree,Draft,"SiteTree"."ID" = 31)
Line 162 of SilverStripeNavigatorItem.php
SilverStripeNavigatorItem_ArchiveLink->getHTML(Page)

This is a real problem for partial cacheing. We had to abandon a training session earlier today. With 4 users accessing the CMS simultaneously, web server CPU usage was at 100% and requests were timing out. Changing all the 'cached' statements to 'uncached' and emptying the cache fixed the problem.

inCharge, nice debugging. Could you post your findings to open.silverstripe.org so that the core devs can keep track of that. Patches to solve that welcome though I wonder what archive link is used for anyway.