Unofficial news and tips about Google

June 14, 2009

Browsers Cache YouTube Videos

YouTube managed to achieve something incredible: browsers now cache YouTube videos and you can load the same video multiple times from the local cache. Try to go to a YouTube video, wait until the video is buffered and then reload the page. You'll notice that the video starts instantly and you can watch it directly from your browser's cache. This works even if you close the tab and open the same page later.

If you have a slow Internet connection or if you frequently watch some videos, the new feature allows you to cache the videos, close the browser and watch the videos at a later time. You may want to increase the size of your browser's cache to preload longer videos. Here are YouTube's recommendations for dial-up connections:

"For best results, you may want to start the video player and then click on the PAUSE button immediately. Then, wait until the red progress bar has reached the end of the video before playing it. Allowing the entire video buffer lets you watch it without interruption."

Videos will load from the browser's cache even if they're embedded in a web page, so this feature is not limited to YouTube's site.

Not quite "incredible", just an artifact of trying to be more standardized.

Flash has been able to use the browser's cache for a long time. In reality, YouTube has explicitly *disabled* caching until now, probably due to filling up browser caches too fast on small hard drives, and content copyright fears.

What will make me smile is if YouTube decides to shed the Flash player altogether for browsers supporting HTML5 video elements. Many content producers are not happy about HTML5 video rendering because it removes the various "protections" that FLV/Flash pretends to offer.

Maybe I'm wrong, but YouTube seems to be the first important video site that plays videos from the cache.

Regarding the transition to HTML5, YouTube has a demo and it will certainly migrate from Flash when most browsers will support the video tag. Google has always said that Flash "lives" outside of the web.

@Anonymous post above mine: I checked your link but it didn't work. I am hoping YouTube will implement HD video playback like that of HQ videos, in that setting up the Playback option will always load HQ videos by default.

This does load the HD video in your link: http://www.youtube.com/watch?v=r0glKrRe4go&hd=1&feature=hd

I've just verified that while a browser will cache YouTube videos to allow for a faster replay, YouTube is still preventing caching of video by an upstream Web cache, thus preventing enterprises and ISPs from accelerating retrieval videos which are viewed by more than one user. Alas, Google has not only failed to achieve anything "incredible," except, perhaps, an "incredible" waste of expensive bandwidth by failing to enable caches to achieve their intended purpose. Back to the drawing board, guys; your system needs work.

I've had a few google people tell me -explicitly- that youtube videos are not to be cached by intermediary proxies.

It didn't stop me doing the Squid hackery to make this possible because, unfortunately for Google, there's a large part of the internet with small amounts of bandwidth between large numbers of people, and having to install Google caching appliances everywhere is a colossal waste of space and power.

As a Squid (and Lusca) proxy cache developer I again put the offer forward to Google/Youtube to work with the open source community and make their content more cachable. I'm even happy to extend the HTTP caching rules in Squid/Lusca to provide Google much more control over revalidation and authentication of movie content. Your users will love you.

If Google isn't allowing caching by "intermediary proxies," it means that YouTube is still preventing caching of its video files by a standards-conforming Web cache (e.g. Squid) operated by a company, an academic institution, or an ISP upstream of the end user. It thus continues to "monkey wrench" these entities, preventing them from avoiding wasteful, duplicative downloads or from accelerating the retrieval of videos which are viewed by more than one user.

Due to this tremendously wasteful practice, ISPs, companies, and universities may be well justified in limiting the bandwidth that YouTube can squander until Google fixes its system. Squid has a feature called "delay pools" which can be used to throttle YouTube. Just add the following to squid.conf:

I would suggest that IT managers, especially at companies and universities, who don't want their bandwidth consumed by YouTube's uncacheable content install these settings until Google stops wasting their bandwidth.

The news is that they've been using a variety of content redistribution methods which end up mapping the same Youtube video to a large variety of potential back-end servers. Your "refresh" on the browser would almost always end up sending the video request to a completely different backend host.

I modified Squid to be extendable enough to collapse all of these backend URLs for a given video to one URL so it could be cached for all users using the proxy server.

Now here's hoping they disable that damned autoplay on profiles, or at least add an option to, because it is one of the next biggest wastes of bandwidth, and an annoyance. (IMO)So many times i have opened a persons profile in the background while watching a video, then suddenly random audio. Then i have to pause current video, go to profile, pause autoplay video, come back to original video and watch, then end up wasting bandwidth on a video (from profile) that i will probably not watch... (it was worse when it was the same video, but not they enabled caching)

I tried to download 'Google Chrome' onto my computor that uses windows XP (as You Tube tell me they will no longer supoport my old browser) and guess what? I can't download any of these new browsers! Why is it that we just can't leave things alone? My old browser worked fine and I could watch You Tube videos fine. Now, they are changing things (what else is new?) and I cannot download a single new browser they promote, so I guess it's "goodbye" to 'You Tube' from me, because I've just about had it with technoligy!

Adrian, thanks for the hack (storeurl_rewrite). I've been using it. I've made a small script to cache Youtube videos based on it. It works perfectly. It is in my webpage, as the explanation (in pt-br).http://lucianopinheiro.net/portal/?q=node/130

This caching doesn't happen in NZ as most ISP have there own cache that they supply youtube from, so if we load a clip and then reload the page, our ISP will send/sell us another copy of that file. I'm sure other ISP's also do this caching to, to save/make them money.

This isn't quite an absolute solution as for example, clicking on related videos appends 'related' to the url of the video. Hence unless you find that video by clicking on related again (from another video), its treated as two separate videos (so your the url in your viewing history is different) and hence cached separately.

Yes, New Zealand has a secondary caching system or they had one anyway. YouTube files are downloaded to that transparently. You actually fetch off that local server for a period after which its automatically deleted.

Anyhow, even if Telecom NZ didn't have this system I don't think you could reliably go back to the same YoutTube URL and have a file reload straight from the Chrome Cache since it could be come off a redirected google/YouTube mirror... as I understand it anyways..

clicked on the video to start playing then immediately clicked it again so it goes to the video's youtube page. this effectively stops the video playback on the page where it is embedded.

then i paused the video and let it download to cache.

once the red bar is full, i went back to the blog page tab and refreshed it. the seek bar did not fill up there. clicked the play button and still the seek bar did not fill up, w/c meant it was trying to re-fetch the entire video.

If you want to cache YouTube video, check out http://www.appliansys.com/VideoCaching

Their CACHEBOX appliance does an awesome job of storing YouTube videos and then serving them to repeat requesters using the same path.

Nearly all YouTube videos can be cached other than where rights are protected. Caching on such an inexpensive device offers significant savings on bandwidth transfer and bandwidth utilisation. This is a perfect solution where transfer is expensive or where the same videos are being watched by multiple users.