Our fine-grained management is single-entry eviction from our code
cache. Since profiling is expensive, the best way to do this is to use
a simple FIFO circular buffer, which has been shown to have miss rates
comparable to more complex schemes like LRU or even LFU.

This replacement strategy means that repeated evictions will replace the
entire cache, repeatedly, so we need to make sure the cache is large
enough. But there's no reason to make it so large that it holds every
instruction the application ever executes: we only need to capture the
working set of the application. The working set varies by application:
for some it's a small fraction of total code size, for others nearly
every instruction executed is hot. This is the heart of our algorithm:
identifying when the cache needs to be resized.