tag:blogger.com,1999:blog-79385110269170479192018-10-14T01:44:02.025+13:00steveblogSteve Bakerhttp://www.blogger.com/profile/07677884151986199976noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-7938511026917047919.post-31449010363669423502013-02-26T10:12:00.000+13:002013-02-26T10:13:55.822+13:00Raspberry Pi as a transparent squid caching proxyDeveloping <a href="https://wiki.openstack.org/wiki/Heat" target="_blank">Openstack Heat</a> means spending a fair amount of time building and&nbsp;customizing&nbsp;bootable cloud images. A lot of this time is spent waiting for RPMs, debs and tarballs to be downloaded by a vanilla guest OS running inside a VM. And given that I work from home with an average broadband connection in a remote country in the South Pacific, the result is some frustrating wait times.<br /><br />Since the same packages are often being repeatedly downloaded, I would benefit from some local caching. This seemed like a good excuse to use a <a href="http://www.raspberrypi.org/">Raspberry Pi</a>. I went with a Raspberry Pi B running <a href="http://www.raspbian.org/">Raspbian</a>. The aim was to set it up as a bridge and run a transparent squid proxy between eth0 (the inbuilt network interface) and eth1 (a USB ethernet dongle).<br /><br />Once I'd completed the initial installation, I installed the following:<br /><span style="font-family: Courier New, Courier, monospace;">$ apt-get install squid3 bridge-utils </span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span>eth0 and eth1 were set up to bridge on my 192.168.1.0/24 network, and an iptables rule was set to direct any port 80 traffic that passes through the bridge to squid's default port. <br /><br /><script src="https://gist.github.com/steveb/5033074.js"></script> The following changes were made to the squid configuration file. Since I'm interested in caching larger files the maximum_object_size has been set to 512MB. My Raspberry Pi is running on a 16GB SD card; for now I have configured cache_dir to use 8GB of that. <br /><script src="https://gist.github.com/steveb/5033062.js"></script> <br /><br />And did this actually help my image building time? Using <a href="https://github.com/stackforge/diskimage-builder" target="_blank">diskimage-builder</a> I ran an Ubuntu customization where the source image file was already cached locally. The first run populated the squid cache with apt repository packages and the second run had a hot squid cache. The build time went from (mm:ss) 04:20 to 01:20 which I'm pretty happy with.<br /><br />Doing the same with <a href="https://github.com/heat-api/heat-jeos" target="_blank">heat-jeos</a> (which is based on <a href="https://github.com/clalancette/oz" target="_blank">oz</a>) managed to get some cache hits on the second run, but had little impact on the (mm:ss) 22:30 build time.Steve Bakerhttp://www.blogger.com/profile/07677884151986199976noreply@blogger.com13