"Trickle Caching" for HD Content Streaming

These days, it's no longer unusual to have 1080P content streamed over the internet, and there are now multiple XBMC plugins that access this content. There is however a problems, in that these streams saturate memory caches. The reason that's a problem, is when the cache is full you can't download anymore and downloading is paused. The reason that's a problem is that there's an latency involved from taking a paused download back to full use of available bandwidth.

This is the one of the reasons people see problems with 1080P streamed content in XBMC despite having internet connections that should handle it. Because XBMC grabs as much as it can right away, it fills the cache up, then stops downloading for a bit, then tries to download again when the cache dips down. If the latency between paused download and full use of bandwidth is too great, then a buffer underrun will occur. This is what causes the skipping in and out of apparently speedy buffering.

This is basically a "Greedy Scheduling" problem. Ideally, you want to always be downloading, so there's no gap between paused download and best use of bandwidth. But if you take everything as fast as it can arrive, then you'll fill the buffer, causing a buffer overrun forcing you to pause downloading. So you're alternating between buffer overruns and buffer underruns.

The way to avoid this is actually pretty simple. Just don't download it all at the highest speed possible, and instead always have a trickle of content coming in to try and constantly be filling the buffer. The classical explanation is filling a bucket with a hole, you don't want to have the tap on full and over flow the bucket, and you don't want to keep turning it on and off because then the bucket will run dry.

The network stack actually does have a lot of clever ways to throttle back downloads, and reduce your bandwidth being used. And the best part is that most stacks do this without having to make any special system calls at all. Just limit how many read() calls are made in any period of time, and the network stack will try to throttle the download to the same rate.

So by rate limiting the number of read() calls, you can keep the buffer in check. Work out a 'sweet spot' in the buffer. Over that, start decreasing the number of read() calls. Under it, start increasing the number of read() calls. Theoretically, you can avoid ever overrunning, and if enough bandwidth is there you should also prevent underruns.

In that case, the cache sweet spot needs adjusting, because it's acting like it doesn't. Perhaps there's simply too little cache to handle 1080p bitrates? Remember we're talking in terms of 3-4mbps streams, so really there needs to be 16mb caches to handle more than few seconds of stream. And that's the lower end. Remember that the ATV3 upgraded their system memory from 128mb to 256mb to be able to handle 1080p streaming, and the bulk of that is for the in-memory cache.

If XBMC has issues with large in-memory caches, perhaps it's really time to start looking at on-disk caching again?

Er... I do not understand the question? I'm suggesting improvement is needed in XBMC's network caching, because it's running up against delivery issues for 1080p bitrates. Do I need to state any more motive than that?

References I read at the moment suggest that the memory cache has issues with being set above 8mb? (Three times as much memory is allocated?)

What are the low-watermark and high-watermark set at?
Sorry, I confused my models when typing.

But... the ATV3 does not share a common SoC with the new iPad. (ATV3 - A5, iPad generation 3 - A5X) It actually only shares components with the Rev2 iPad 2, and only the dieshrink A5. But I'm not sure why you're arguing this, surely it's a given that higher bitrate content needs revision of assumptions about memory caches?

(2012-05-07 22:01)barberio Wrote: Er... I do not understand the question? I'm suggesting improvement is needed in XBMC's network caching, because it's running up against delivery issues for 1080p bitrates. Do I need to state any more motive than that?

I'm all up for improvements, but I can't improve anything if I don't know what the problem is in the first case.

(2012-05-07 22:09)barberio Wrote: References I read at the moment suggest that the memory cache has issues with being set above 8mb? (Three times as much memory is allocated?)

What are the low-watermark and high-watermark set at?
Sorry, I confused my models when typing.

But... the ATV3 does not share a common SoC with the new iPad. (ATV3 - A5, iPad generation 3 - A5X) It actually only shares components with the Rev2 iPad 2, and only the dieshrink A5. But I'm not sure why you're arguing this, surely it's a given that higher bitrate content needs revision of assumptions about memory caches?

And where do you think the sram is located on an A5 Hint, it's on chip, the A5 is a die stack.

Well, I do notice there seems to be issues that xbmc's default cache stategy results in 'bouncing' in and out of cache for streams when there is ample bandwidth available. This is pronounced when it's a source that had high bandwidth but high latency, which is exactly what I would expect for hitting buffer overruns too much.

Can I check, is the default stream cache 5mb as the wiki documentation suggests? I've just changed that to 64mb in advancedsettings.xml and tested with a source that was showing the problem, and it seems to no longer get the cache 'bouncing', suggesting that the current default is too low for usable cache, or the high watermark is too high. Is there a hard-limit to this cache size, and what is it?

(2012-05-07 22:30)davilla Wrote: And where do you think the sram is located on an A5 Hint, it's on chip, the A5 is a die stack.

This really is nothing to do with what we're discussing. But again, the new iPad that they're producing in huge numbers doesn't use the new die-shrink A5 in the ATV3. It uses the A5X which is an entirely different component. The A5 dieshrink is being used in the low volume rev.2 iPad2, but that's more because they're not making the original A5s any more.

Whatever, you brought it up "Remember that the ATV3 upgraded their system memory from 128mb to 256mb to be able to handle 1080p streaming", ATVs never had 128MBs of ram, so your example is bogus as an ATV2 is perfectly capable of handling 1080p streaming as is. It's the greater memory bandwidth from A4 to A5 that has allowed Apple to enable 1080p decode on the ATV3.

3-4mbps streams. Thats not quite high bitrate. Even XBMC on the PI can handle 10mit streams with his 100MBit usb networkdevice. It smells like a networkproblem in your setup. Against a bad setup we can't do anything.

And as I said, I typed the wrong memory sizes for the models by accident. Now look, at the moment you're just in this thread now to pick a fight, and disrupt a discussion on cache streaming sizing. If you have anything to contribute to what the default memory size, low-watermark and high-watermark should be, please do so.

Having toyed around, I think that with the amount of 1080p streamed content now available, 64mb with a low-watermark of 30% and a high watermark of 60% would be better than the current apparent default of 5mb and an unknown low/high-watermark. If the cache is too small, trickle-feed when over the high-watermark does nothing to help. I'd even consider 128mb as the default if on a suitable system.

Is it feasible to have a default that depends on identifying the available system memory?

(2012-05-07 22:57)gimli Wrote: 3-4mbps streams. Thats not quite high bitrate. Even XBMC on the PI can handle 10mit streams with his 100MBit usb networkdevice. It smells like a networkproblem in your setup. Against a bad setup we can't do anything.

That's local network low-latency traffic, which isn't a problem because the delay added from a buffer overflow is insignificant. It becomes a problem when you're getting the stream from a higher-latency WAN source.
Also, just run some tests. Sourcing video from 'Crunchyroll' (Using the Crunchyroll Takeout plugin) that I know comes from a source that has the bandwidth to deliver to me but if from a different continent so has high latency. With default caching, the 1080p stream keeps bouncing into 'caching' every twenty seconds or so. With caching increased to 128mb (or what ever the hard coded limit is), I can play back the from the same source without a 'caching' pause during playback for the entire 24 minute episode.

So no, not a problem with my networking. A problem with XBMC's caching defaults.

we understand that our current caching policy does not fit all users, we welcome patches to resolve this but to be fair, just increasing the amount of caching will not resolve the issues for all users. We have been there, done that already. See advancedsettings.xml and there are settings there to alter the caching.