Saturday, January 28, 2012

P wanted some firewall rules on Cyberoam so that the live video broadcasts are not affected by people browsing, downloading, etc.

1. Added a schedule under
Firewall -> Schedule -> Define Schedule
called LiveTimes and added the times 7.50 am to 10 am and 4.25 pm to 6.30 pm.

2. Created a Bandwidth policy under
Policies -> Bandwidth Policy -> Create Policy
called LiveTimesThrottle to throttle bw to 256 kbps up and down.
It says Bandwidth in KB - so we have to enter 256/8 = 32 in the box there.

3. Added the LiveTimes schedule for this policy.

4. Added it to Firewall rule in Firewall -> Manage Firewall -> Rule 1 Any Host to Any Host Accept.
Now I am not sure if the Voxel traffic also will be allowed! So, adding a firewall rule,

Unfortunately, this had the opposite effect to what was desired - during the live times the bandwidth was unlimited, and at other times the bandwidth policy was enforced! This was because the bandwidth policy page says:Default Values to be applied all the time....
and the schedule is added underAdd Schedule wise details to override default Bandwidth details

So one way out would have been to reverse the schedule timings. But P wanted an alternate solution, just giving priority to the RTMP streams to and from Voxel instead of throttling all traffic on all days at those times. So, he decided on creating a bandwidth policy, Guaranteed 1 Mbps (128 KB) and Burstable 2 Mbps (256 KB) for RTMP service to voxel, with maximum 0 priority. Now we have to check whether enabling just this bandwidth policy is enough by itself, or we need to explicitly de-prioritize other traffic.

Wanted the home page to be not cached, since they update it often. Looking into it, found two interesting articles, at stackoverflow and metaskills. The first article gave a tip on how to control browser cache for specific urls - using the LocationMatch tag. The second one recommended expiring all html pages immediately. I followed the second one, since our server is not facing any resource crunch right now.

ExpiresActive On

ExpiresDefault "access plus 1 seconds"

ExpiresByType text/html "access plus 1 seconds"

ExpiresByType image/gif "access plus 120 minutes"

ExpiresByType image/jpeg "access plus 120 minutes"

ExpiresByType image/png "access plus 120 minutes"

ExpiresByType text/css "access plus 60 minutes"

ExpiresByType text/javascript "access plus 60 minutes" ExpiresByType application/x-javascript "access plus 60 minutes" ExpiresByType text/xml "access plus 60 minutes"
Initially this block in apache2.conf was inside an if tag, IfModule mod_expires.c - but was not working. Then I commented out the IfModule tags - then it started working, since anyway I had enabled the module using a2enmod expires

Did not add the other two plugins, for city and ISP, since that would slow down AWstats.

Had to flush newer records, by deleting files in DataDir - /var/lib/awstatsin order to process older log files, as given athttp://awstats.sourceforge.net/docs/awstats_faq.html#OLDLOGTook around 15 minutes to crunch old logs for 15 days, so probably the log for 3 hours will take only half a minute or less.

Next for Wowza. Did not have to modify the logging parameters as given athttp://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/
since most of the parameters were already set. But had to change the LogFormat drastically, since my Wowza version is different - 3.03. Also, one point which he makes is to log only the destroy stream event. Not doing that, logging everything as we are doing, was causing over-counting of bytes sent. So, changed the LogFile parameter to readLogFile=”/usr/local/awstats-wmc/tools/logresolvemerge.pl /usr/local/WowzaMediaServer/logs/*access* | grep destroy |”

Sunday, January 22, 2012

Tested the APC Smart-UPS 5000VA with our projector and sound system load. Lasted for 45 minutes with 2 battery LEDs remaining, out of 5. Charging without load, with output switched off, came back to 5 LEDs within 8 hours.

Saturday, January 21, 2012

This part of the configuration, for the audio streams, was a series of experiments with Shoutcast v1 and v2 and Icecast2 - from Dec 15 2011 till now. Duplicating our Shoutcast server setup using v1 server as on our earlier server could have been done, but P was keen on improving things, especially the recorded to live (and back) transitions.

Shoutcast2 was our first trial, since using it would help us to restore our multiple streams on shoutcast.com directory. Tried Shoutcast2's multiple mount strategy. The Shoutcast2 source sc_trans does re-encoding, so we don't want that. Then we found that as this forum post indicates, sc_serv2 does not work with 1.x source like ices - so we need to have a Shoutcast1 server between ices and Shoutcast2. And when that happens, we can't use many of Shoutcast2's features. Also, if we wanted it running on multiple ports, we would need unique mount points on each port - all the ports show all the mount points. Then came the killer - Shoutcast2 was dumping listeners at live/recorded changeover even with backupfile set, and autodumpusers=0 , when relaying the Shoutcast1 streams. So that ended our Shoutcast2 experiment.

When fallback override is set to true, whenever a source connects to /livemount, the listeners are automatically moved from /recordedstream to /livemount. And automatically moved back when the source disconnects. Very nice.

But preliminary exploration gave some disappointments - multiple ports still showed up all the mounts on all the ports, and the fallback mounts did not work for shoutcast relays. Also, there were some funny (buggy?) aspects, like mount points with relays not showing up on the admin interface till a (non-relay) source was connected. Also, how to configure the fallback mechanism for TeluguStream was not clear, since we would not be connecting a source to it directly. The fallback update mechanism given in the icecast admin section did not work, since there was no source directly connected to that mountpoint. Finally, cobbled together the following method.

lots of alias directives, one for each port, redirecting the root / to the correct stream for that port. This could be done because aliases can be defined on a per port basis.Aliases like alias source="/" port="8008" dest="/discoursestream"

TeluguStream falls back by default to its recorded stream, and AsiaStream falls back first to the live stream mount, and then to recorded stream - so the live changeover for AsiaStream is taken care of.

An additional mount was defined as a relay of asiastream, and another icecast config file was written where TeluguStream falls back to this relay instead.

When TeluguStream has to go live, the icecast config file is switched over, andkillall -HUP icecast makes icecast reload the config file - so, listeners who connect now fall back to the relay of asiastream .

Using the admin interface, the listeners who were already connected to the recorded telugustream mount are now moved to the relay mount.

These steps were written into a shell script, with wget being used to do the admin interface calls - wget takes --user= and --password= parameters.

For our website's jplayer-based web player to work, had to change the called URL - for shoutcast it was mp3:"http://stream.radiosai.net:8002/;stream/1" and now for icecast it should be mp3:"http://server1.radiosai.org:8002/"The stream suffix was for Shoutcast to serve the stream instead of the status page. Our aliases are configured to redirect requests from root to the appropriate stream, so we can just request the root, /

Just a caveat - icecast by default has very small numbers in threadpool and source connections. Have given 50 and 20 for now.

Made icecast run like a daemon by calling it with nohup:nohup icecast -c /path/to/icecast_config_file.xml > sh.out 2> sh.err < /dev/null &

Then came the monitoring part. P pointed to amptracker.com with nice graphs. But they wanted $5 per month for the graphs. So, went back to shoutstats, which now has support for icecast too. But after install, found it did not work. Troubleshooting:

First, apache was serving the file index.php like html, without rendering the php parts. Reason was, php short tags were turned off in our server, as found with phpinfo(). Enabled them in /etc/php5/apache2/php.ini
and restarted apache2, the page displayed now. But shoutstats itself was not showing any graphs, even after I updated manually by accessing update.php manually.

Enabling print statements for debugging inside the php pages, found that though rrdtool was being called, it was not creating the .rrd file. File permissions were 777 for the relevant directories. Finally, printing out the exact system() call pointed out the error - there was a missing space after $rddtool in the system call. ' create '.$rddfile.'DS:audience...should have been ' create '.$rddfile.' '.'DS:audienceMentioned it in the github page, fixed it, shoutstats working after correct cron job entered.

Had to remove the hidden status of the mounts - otherwise shoutstats would not work out of the box, would have to change the parsing of status2.xsl. So, made all the mounts visible - hidden = 0.

Most probably this finishes the server configuration posts. Possible tweaks needed for icecast would be to make the logs smaller - maybe - and maybe some tweaking needed for thread management under load and so on.

Wednesday, January 18, 2012

Struggled for a while with mod-rewrite and all that to redirect one page to another page on the apache server. Finally solution turned out to be simple, as referred to in one line in the page above,
"Note that this can also be handled using the RedirectMatch directive". Syntax was, in relevant virtual host file, within the Directory tag, RedirectMatch path/to/url.html http://server.path/to/redirected/file.htm

Monday, January 16, 2012

Disabled ssh with password using the procedure in the earlier post, copying required keys to .ssh/authorized_keys

Started copy process with sftp from colinux, running under screen from saiwaves.

Enabled vsftpd -

service vsftpd start

To make it useful, had to enable ftp write by

vim /etc/vsftpd.conf

and un-commenting the line

write_enable = YES

Also, added username1 user to /etc/ftpusers - so username1 is not allowed to ftp (only username2 is allowed to ftp)

Copied over the entire web folder from old server. Tried this first with Windows Explorer's built-in ftp, but was taking too long - several days - and breaking too often. So did it with Filezilla and completed in 10-15 hours.

Made postfix work: it was giving the error send-mail: fatal: open /etc/postfix/main.cf: No such file or directoryCan't send mail: sendmail process failed with error code 75Solution was, as given here, dpkg-reconfigure postfix

Worked out why and how the metadata is not sent for our shoutcast streams by ices: the song info is not displayed if the SrcIP=ANY and DestIP=127.0.0.1 and the ices connects on an address other than 127.0.0.1. Basically, song metadata is not sent if the ip on which ices connects does not have a route to one of the ips in DestIP.

For ices not to show the filenames for icecast server, the above trick does not work. Also, the method mentioned here and here, of setting mp3-metadata-interval to 0also does not work. So, fiddled with the ices source code and recompiled. Just commented out most of the code in metadata_update function. No more metadata!

Enabled virtual hosts as given here, first making a suitable file sitename.com in/etc/apache2/sites-availableand then a2ensite sitename.com
service apache2 reload

Our windows web server was ignoring case in filenames - uppercase or lowercase did not matter.
With the new Linux server, case does matter, and many many links break.
To fix, loading mod_speling.
It was already available in
/etc/apache2/mods-available.
So only had to do
a2enmod speling
(a2enmod = apache2 enable module script available on debian/ubuntu)
Then, had to add
CheckSpelling on
in /etc/apache2/apache2.conf(This part is not well documented!)

Had a large number of Main.asp files (for each old journal issue cover page). Actually these files were almost completely html. So, took the short-cut of serving these files as html. Edited the relevant file in sites-available to add

Then added DirectoryIndex Main.asp
This caused major breakage, till I added all the other default index pages!DirectoryIndex Main.asp index.htm index.html index.php

Apache was giving a 403 forbidden for most pages. Turns out it needs execute permissions also on the directories. This page discusses the same problem and the solution. So, set local_umask=022file_open_mode=0777in vsftpd.conf

The configuration for the radio streams is yet to be finalized, since the icecast configuation has to be tested.

I'm back to the iPhone - the farewell did not work out! Selling it on ebay did not work so well, as the highest bidders seems to be phoneys who bid higher than the Buy Now price, and then did not pay up. Also, the touchscreen on the windows mobile stopped working, making it lose a lot of functionality, like zooming pdfs and images. Showing IRCTC and KSRTC tickets on the mobile then becomes difficult :)
So, back to iPhone.

Unfortunately, I overwrote Stanza with the newer version, which crashes for old versions of iOS. Installing old versions of Kindle and Stanza with iFunBox did not work - the apps always crashed. Researched and found: