What is LUSCA/SQUID ?

LUSCA is an advance version or Fork of SQUID 2. The Lusca project aims to fix the shortcomings in the Squid-2. It also supports a variety of clustering protocols. By Using it, you can cache some dynamic contents that you previously can’t do with the squid.

Advantages of Youtube Caching !!!

In most part of the world, bandwidth is very expensive, therefore it is (in some scenarios) very useful to Cache Youtube videos or any other flash videos, so if one of user downloads video / flash file , why again the same user or other user can’t download the same file from the CACHE, why he sucking the internet pipe for same content again n again?
Peoples on same LAN ,sometimes watch similar videos. If I put some youtube video link on on FACEBOOK, TWITTER or likewise , and all my friend will watch that video and that particular video gets viewed many times in few hours. Usually the videos are shared over facebook or other social networking sites so the chances are high for multiple hits per popular videos for my lan users / friends.
This is the reason why I wrote this article. I have implemented Ubuntu with LUSCA/ Squid on it and its working great, but to achieve some results you need to have some TB of storage drives in your proxy machine.

Disadvantages of Youtube Caching !!!

The chances, that another user will watch the same video, is really slim. if I search for something specific on youtube, i get more then hundreds of search results for same video. What is the chance that another user will search for the same thing, and will click on the same link / result? Youtube hosts more than 10 million videos. Which is too much to cache anyway. You need lot of space to cache videos. Also accordingly you will be needing ultra modern fast hardware with tons of RAM to handle such kind of cache giant. anyhow Try it
We will divide this article in following Sections1# Installing SQUID / LUSCA in UBUNTU2# Setting up SQUID / LUSCA Configuration files 3# Performing some Tests, testing your Cache HIT
4# Using ZPH TOS to deliver cached contents to clients vai mikrotik at full LAN speed, Bypassing the User Queue for cached contents.

1# Installing SQUID / LUSCA in UBUNTU

I assume your ubuntu box have 2 interfaces configured, one for LAN and second for WAN. You have internet sharing already configured. Now moving on to LUSCA / SQUID installation.
Download LUSCA source and compile it using,

All of Lusca/Squid configuration files can be found at
/usr/local/squid/etc/
and squid executable can be found at/usr/local/squid/sbin/
Now We will edit squid.conf file to make it customize according to our requirements by . . .

1

nano /usr/local/squid/etc/squid.conf

Delete all previously lines , and paste the following lines.
Remember following squid.conf is not very neat and clean , you will find many un necessary junk entries in it, but as I didn’t had time to clean them all, so you may clean them as per your targets and goals.

001

# SQUID / LUSCS TEST CONFIG FILE

002

# Email: aacable@hotmail.com

003

# Web : http://bolozer.com

004

005

# PORT and Transparent Option

006

http_port 8080 transparent

007

server_http11 on

008

icp_port 0

009

010

# Cache Directory , modify it according to your system.

011

# but first create directory in root by mkdir /cache1

012

# and then issue this command chown proxy:proxy /cache1

013

# [for ubuntu user is proxy, in Fedora user is SQUID]

014

# I have set 10 GB for caching reserved just for caching ,

015

# adjust it according to your need.

016

# My recommendation is to have one cache_dir per drive. zzz

017

018

#store_dir_select_algorithm round-robin

019

cache_dir aufs /cache1 10000 16 256

020

cache_replacement_policy heap LFUDA

021

memory_replacement_policy heap GDSF

022

023

# If you want to enable DATE time n SQUID Logs,use following

024

emulate_httpd_log on

025

logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt

026

log_fqdn off

027

028

# How much days to keep users access web logs

029

# You need to rotate your log files with a cron job. For example:

030

# 0 0 * * * /usr/local/squid/bin/squid -k rotate

031

logfile_rotate 14

032

debug_options ALL,1

033

cache_access_log /var/log/squid/access.log

034

cache_log /var/log/squid/cache.log

035

cache_store_log /var/log/squid/store.log

036

037

#I used DNSAMSQ service for fast dns resolving

038

#so install by using "apt-get install dnsmasq" first

039

dns_nameservers 127.0.0.1 192.168.2.4 192.168.2.3

040

ftp_user anonymous@

041

ftp_list_width 32

042

ftp_passive on

043

ftp_sanitycheck on

044

045

#ACL Section

046

acl all src 0.0.0.0/0.0.0.0

047

acl manager proto cache_object

048

acl localhost src 127.0.0.1/255.255.255.255

049

acl to_localhost dst 127.0.0.0/8

050

acl SSL_ports port 443 563 # https, snews

051

acl SSL_ports port 873 # rsync

052

acl Safe_ports port 80 # http

053

acl Safe_ports port 21 # ftp

054

acl Safe_ports port 443 563 # https, snews

055

acl Safe_ports port 70 # gopher

056

acl Safe_ports port 210 # wais

057

acl Safe_ports port 1025-65535 # unregistered ports

058

acl Safe_ports port 280 # http-mgmt

059

acl Safe_ports port 488 # gss-http

060

acl Safe_ports port 591 # filemaker

061

acl Safe_ports port 777 # multiling http

062

acl Safe_ports port 631 # cups

063

acl Safe_ports port 873 # rsync

064

acl Safe_ports port 901 # SWAT

065

acl purge method PURGE

066

acl CONNECT method CONNECT

067

http_access allow manager localhost

068

http_access deny manager

069

http_access allow purge localhost

070

http_access deny purge

071

http_access deny !Safe_ports

072

http_access deny CONNECT !SSL_ports

073

http_access allow localhost

074

http_access allow all

075

http_reply_access allow all

076

icp_access allow all

077

078

#==========================

079

# Administrative Parameters

080

#==========================

081

082

# I used UBUNTU so user is proxy, in FEDORA you may use use squid

083

cache_effective_user proxy

084

cache_effective_group proxy

085

cache_mgr aacable@hotmail.com

086

visible_hostname proxy.aacable.net

087

unique_hostname aacable@hotmail.com

088

089

#=============

090

# ACCELERATOR

091

#=============

092

half_closed_clients off

093

quick_abort_min 0 KB

094

quick_abort_max 0 KB

095

quick_abort_pct 100

096

vary_ignore_expire on

097

reload_into_ims on

098

pipeline_prefetch on

099

range_offset_limit 512 KB

100

read_timeout 30 minutes

101

log_fqdn off

102

memory_pools off

103

104

# If you want to hide your proxy machine from being detected at various site use following

105

via off

106

107

#============================================

108

# OPTIONS WHICH AFFECT THE CACHE SIZE / zaib

109

#============================================

110

# If you have 4GB memory in Squid box, we will use formula of 1/3

111

# You can adjust it according to your need. IF squid is taking too much of RAM

3# TESTING YOUR CACHE HIT

It’s time to hit the ROAD and do performing some tests.

YOUTUBE TEST

Open Youtube and watch any Video. After complete download, Check the same video from another client. You will notice that it download very quickly , you can watch the bar moving fast.As Shown in the example Below . . .
monitor the SquidaccessLOG. You will see cache hit TPC_HIT for this video.As Shown in the example Below . . .

MUSIC DOWNLOAD TEST

Now test any music download. For example Go tohttp://www.apniisp.com/songs/indian-movie-songs/ladies-vs-ricky-bahl/690/1.htmlAs Shown in the example Below . . .

and download any song , after its downloaded, goto 2nd client pc, and download the same song, and monitor the SquidaccessLOG. You will see cache hit TPC_HIT for this song.As Shown in the example Below . . .

EXE / PROGRAM DOWNLOAD TEST

Now test any .exe file download.
Goto http://www.rarlabs.com and download any package. After Download completes, goto 2ndclientpc , and download the same file again. and monitor the Squid access LOG. You will see cache hit TPC_HIT for this file.As Shown in the example Below . . .SQUID LOGS
More to come , , ,

Posted by djoko oetomo on Jumat, 10 Februari 2012 - Rating: 4.5

Title : Howto Cache Youtube with SQUIDDescription : What is LUSCA/SQUID ? LUSCA is an advance version or Fork of SQUID 2 . The Lusca project aims to fix the shortcomings in the Squid-2. I...