A.b - MythTV HD Master Backend / Frontend

AMD X2 5000+ with 4GB (32 bit)

Asus M4A79 Deluxe with Asus Geforce 210 1GB DDR3 (using VDPAU)

With VDPAU, HD 1080p works flawlessly at 1920x1080. The X2 2.6Ghz AMD could almost play 1080p by itself but stuttered. The AMD X2/Geforce 210 replaced an Intel Dual Core 3Ghz (no VDPAU or XvMC) that made HD watchable, but it had infrequent issues with stuttering.

Hauppauge_HD-PVR and uses 6200ch to change channel through firewire connected to DCT-6200

Roughly 1.2TB in a Raid 5 setup and 1TB LVM for video backup (rsync'd each night with master server)

Both SD and HD 720p/1080i work great under VMware Workstation! (still) stutters but it at least %90 watchable now. (using extra audio buffering and libmpeg2 with xv-bit fixed it) The stuttering is size dependent - If the VMware guest is expanded to the maximum size of the monitor, 2560x1600, it stutters, but if a border of about 600 pixels is left around each side of the VMware guest, roughly 1080p, it works fine.

Here is a picture of the setup - Which is out of date now, there are now five portrait monitors, four on the Ergotron Triple Display, one on a separate mount and a horizontal 30" DELL on it's regular stand to the side.

System B

This system lives at my brother's house although I maintain it and it is a separate system.

Notes

The link between the MythTV frontends B, D and the master and slave backends A, C is over a 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and one 21dbi and one 9dbi gain antenna. Average throughput for 1080p HD is about 13mbps and during high motion scenes, tops out at 19mbps. With the high gain antennas, the TEW-510apbs can operate at their maximum throughput of 21Mbps and HD is smooth. On rare occasions, the link is affected by thick ice on the roof and really heavy rain. (System B is now separate)

Given three of the systems (A.a, A.b and B.b) are on 24/7, they each have a UPS rated at 900VA or better, which lasts an average of twenty minutes before cleanly shutting down (important for the RAIDed drives).

We currently use link2voip.com The Spout (.ca) as our DID and outbound provider for Asterisk.

Most of our machines use gigabyte Ethernet and both systems use Linksys WRT54GL variants running the open source firmware, DD-WRTTomato (Tomato isn't updated enough and has some intermittent bugs). There have been a lot of gateways in this house but the open source DD-WRT is by far the best. It handles QoS properly for VoIP and has all the bells and whistles every other gateway lacks. We also use OpenDNS for all the DNS queries.

We use most of the default plugins including MythGame (Who am I kidding, kids game on iPhone/Pads these days), pictures (rarely), videos, weather and XYAC for CallerID.

Both Asterisk boxes are connected two each other over an IAX2 link that works well through NAT and our mutual ISPs.

On System A, we also run two TrendNet (TV-IP252 and TV-IP422W) cameras for security that record any motion detected to A.a

Tips from the HD Front Lines

I have been using MythTV with HD now for a couple years and wanted to share what has worked and what has not. These comments could save you a lot of time and frustration:

Firewire cards - get the best firewire cards that works with your tuner(s). I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards. Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes. Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast! You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.

VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks. VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time. Recordings seem to work %100 of the time. Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged). I have no clue why because the same combination but with a DCT-6214 worked fine. The tuner hard lock also doesn't happen on HD channels.

Conclusion - With DCT-62xx tuners, get Agere-5t FW323 v61 firewire chips, dump the priming script and use Broadcast at 200mbps.

Video cards - As XvMC support is going to be discontinued in MythTV 0.25 and replaced with VPDAU which supports a much large set of codecs. NVIDIA has perhaps the best support and look for a card that uses "PureVideo HD" and perhaps the cheapest is the GeForce 210 which works great.

Conclusion - Get a Geforce 210 based video card and HD will work flawlessly with the OSD using VDPAU.

Processor - Get the largest and fastest one for HD because it makes a difference. Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding. A faster processor also helps with database queries. (As an aside, I also suggest looking into keeping the database in memory)

Conclusion - Faster = better.

Remote Control - From a usability stand point, an excellent remote is important. I have settled on the Microsoft MCE remotes because they have lots of features that make it all come together - First, they can be trained with your existing TV remote to shut off the TV and control the volume of the TV. The volume control in MythTV frankly sucks because it only works when a recording is actually playing and doesn't go high enough (a fault of the hardware actually). Being able to turn off the TV and control it's volume directly mean you can chuck your TV's remote. If you are playing DVDs on your MythTV system, you can now have one remote to control everything and that rocks. Better, if you wreck the remote, they are inexpensive ($35 locally) and widely available. The receiver has wide reception, a quick response rate and the layout on the remote is excellent with important buttons in easy to reach places. The downside is that because it is back lit, it uses batteries quickly (every four months) but hopefully rechargeable ones in your case.

Conclusion - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box. One remote to control them all.

Hard drive Size - On the main backend (A) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed. I tried RAID 5 but ... [UPDATE] I have switched back to RAID 5 and it seems to work fine - best use the latest kernel to get all the latest updates. (Others have reported success with RAID 5) Having said that, the slave backend (C) [UPDATE] now has RAID 1 redundancy and records HD to two SATA II drives which seems plenty fast enough.

Hard drive size really only translates into one thing - longer time to watch something before it expires. For just recordings, our system has a combined space of about a terabyte which means our recordings last about a month. Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows. Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually). That harddrive is then RSYNC'd each night from the main server (A) to extra space on the slave backend (C).

Conclusion - Larger = Longer lasting. RAID is your friend.

Keeping HD In Sync When Transcoding

One of the benefits of MythTV is that once the shows are recorded, they can be manipulated in many different ways, for example, the commercials stripped out and the rest dumped onto a DVD or transcoded with an open source codec like Xvid. Transcoding HD recordings from either over-the-air or firewire captures presents significant difficulties, mostly related to keeping the audio in sync with the video.

I have found, through experience, that an MPEG-2 TS file fresh from being recorded over firewire from a DCT-62xx tuner, and edited with something like Avidemux2, will result in an audio sync that will get worse as the recording plays. By the end of the recording, what started as a 250ms sync, is now out by a couple of seconds.

There are two problems related to keeping the audio and video in sync, the first are errors in the stream, which over time create small differences. Unlike VLC which constantly adjusts the sync based on time codes, editors like Avidemux do not. The other problem is commercials, which not only interrupt the MPEG-2 stream, but do other nasty things like change framerates and drop to lower resolutions.

Looking over hundreds of forums, the consensus has been to pre-process the stream with ProjectX, an open source Java utility, which, for the longest time, worked. I would edit out the commercials with ProjectX's slow interface, demux the final cut, load it into Avidemux, load the filters and transcode. I like Avidemux because it is open source, has a slick interface and rips to a wide array of formats with lots of filters, like cropping, deinterlacing, framerate and resizing. Moreover, it's greatest feature is that it runs on both Windows and Linux which means the many hour transcodes can occur, as a background process, on our 24/7 Linux boxes. Unfortunately, Avidemux happens to be terrible at decoding raw MPEG-2 streams and isn't the most stable of programs.

However, ProjectX has recently not been able to deocode the MPEG-2 streams nor can repair utilities like mpeg2repair fix them. I was stuck.

The great part is that another MythTV user created a script to rip HD, that, for some reason, keeps the audio in sync. Unfortunately, the script only works on raw MPEG-2 streams which means it doesn't run on the transcoded commercial free recordings that MythTV can create. Besides doing the commercial detection automatically, MythTV has a fast interface for fixing the cut points and it is suppose to be able to do a straight lossless MPEG-2 to MPEG-2 copy. Unfortunately, because of a long outstanding bug with mythtranscode, it doesn't work.

If mythtranscodes's lossless copy did work, the whole process could be automated with a few clicks of the remote:

Manually adjust any cut points (if necessary) from what MythTV has detected

Tell MythTV to do a lossless MPEG-2 to MPEG-2 transcode which removes the commercials

Tell MythTV to run the user HD script to make an Xvid archival copy, dumping the result in a certain spot to be renamed and moved into an appropriate directory later.

Without step 2, you have to run the HD script on the raw recording with the commercials included, load the result into Avidemux and manually find and cut the commercials. There is still an audio video sync problem, but I have found that whatever mencoder does, the audio delay is a straight 250ms delay, consistent from recording to recording and constant across the entire recording. The other good news is that if you don't have to crop anything, the Xvid to Xvid copy is fast, taking less then a minute for a two hour long show.

Conclusion - if you want to transcode an HD recording that you cannot edit with ProjectX use the following process:

I had to modify the script to accept larger files and create a smaller end result (640x360). I can post the modified script here if anyone is interested.

This transcode takes a long time, upwards of six to eight hours on a Pentium 3Ghz machine.

Load the resulting file into Avidemux, leave the audio and video as "copy", cut the commercials and set the audio delay to 250ms

Save it and hit "yes" to smart copy. Sixty seconds later you have a perfectly transcoded and in sync result.

Other Random Tips

FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster. PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.

Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer. The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.

Monitoring of the Myth systems work great using Gkrellm and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins.

Personal Setup Notes

Notes for my own setup:

Distinctive Ringing

Setting up distinctive ringing for ring groups.
1. Via A@H or FreePBX, create two ZAP extensions for each ZAP channel,

a. The lower number one, i.e. ext 201 uses a "dial" command "ZAP/1r2", where 1 is the ZAP daughter card in slot 1 and "r2" stands for ring type 2 (or r3 or r4). This extension will ring the phone using the distinctive single/long ring.
b. The second ZAP extension, i.e 211, uses just "ZAP/1" as the dial command and will ring the same ZAP phone with a regular US ring tone.

2. Then set it up such that internal callers go to ring group 200 which rings all the distinctive ZAP extensions, i.e. x201, etc. When external callers call, their separate ring group calls x211, etc. with regular ring commands.

The great thing is that the modifications are done through A@H and don't need custom modification of any script.

To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402

Caller ID

Code for sending caller ID from Asterisk to MythTV Frontends. Requires XYAC. Use the startup script below to start up the XYAC daemon at login.

The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls. Without the "&", the far side can intermittently time-out just as the phones start ringing.

Idle Three Hour Sleepd Daemon

Below is a method of using the sleepd daemon to shut off a MythTV frontend after three hours and to pop up a 30 second warning that the user can cancel.

Backend Slave and Master MythTV Backend Process Reset

The WAF had been lower of late mostly because our master/save system seems to hang once in a while. I added a method whereby the master, slave backends and the frontend are reset at the push of a remote button. A local script is run when the button is pushed that does the following (call it "restart_entire_myth.sh"):

I'm not sure the "-t" is necessary now, as I was using it for debugging purposes.

To get that SSH command working, you need to have a properly working SSH link between the computers that doesn't require a password, but uses a public key authentication mechanism. There are lots of resources on the net to set this up and one useful one is here.

You also need to allow a regular MythTV user to restart the processes on both the master and slave backends. With root privileges on each, run visudo and add at the end of the file

<username> <hostname>= NOPASSWD: /sbin/restart mythtv-backend

Where <username> is the user your are either login into via SSH or under which you are running the MythTV frontend. You can test the NOPASSWD setting by exiting root privileges (#exit) and then using "sudo -k" to remove the root access. Then try restarting the mythtv-backend process ($restart mythtv-backend) If the process reset correctly without root access, then it will work for the local or SSH user.

Once you can then run the "restart_entire_myth.sh" script mentioned above to see if actually restarts all the appropriate processes. Once the restart script works, then it is necessary to tie it into the power button on the remote. The first step is to modify the .lircrc file that is used by MythTV to make sure it passes the command to run to "irexec"

The directory where "restart_entire_myth.sh" lives is in the path used by the OS to check for files (can't remember how to set this up), but worst case, use the direct path (i.e. /home/my_custom_scripts/restart_entire_myth.sh). The next tricky bit is to get irexec to take input from the right screen. I also had issues with irexec not working unless it was run in a TTY capable terminal (no idea why). To get around these issues, I ran irexec via a script that run when the MythTV user logged in:

The "-H" is probably not necessary and just holds the window open, but the "-x" runs the "irexec" command. You can also run this script and test to see if the button press from the remote is getting caught and running the restart script, the output of which appears in the terminal window. I use xfce desktop obviously, but I'm sure the same can be done for the Gnome and KDE terminal programs.

The Plus and Minus of MythTV

Note this was a comment added to an article comparing TIVO 3 and MythTV.

I've been a MythTV user for a couple years now and there are a number of things the article missed or glossed over ("+" means a feature and "-" a problem).

(+) Plugins - MythTV has lots of plugins that let you do everything from surf the web, view local weather information, play old style ROM arcade games, watch recordings over the LAN on a windows machine, view a slide show of pictures, to watching XviD encoded videos, all of which we use. The video collection has been of great utility with kids. The usefulness of the plugins was a big reason I chose MythTV and have stayed with it. (Actually TIVO was never considered because of the subscription costs)

(-) Setup and Maintenance - It should not be underestimated how much time it can take getting MythTV to work and then maintaining it. As a MythTV user you will have to know the Linux command line, how to download and install drivers and fix things when they inevitably don't work. If you haven't installed a PCIe card before, and don't like googling stuff to figure out why it doesn't work, then MythTV is not for you. MythTV will probably not save money over a commercial system either because I've found that the money I would have spent on a subscription went into hardware upgrades.

(+) The flip side to the cost and time commitment is that MythTV has immense flexibility, for example, in the size and redundancy of storage capability, the type of peripherals and the split arrangement between backend, slave backend and frontends. Take our ever growing video collection (most ripped from DVDs, fyi), where any new videos are backed up onto a separate machine each night (rsync'd). Try doing that with a TIVO.

(+) The machines are also full blown general purpose computers that can do other things. For example, once you have sunk the money into the MythTV hardware, you can use the same machines for VoIP (via Asterisk) which really saves us money, about $100CND a month. The pay back period for the extra hardware to do VoIP was less then a year, and gets cheaper as time goes on. We use dedicated Digium TDM-400 cards that cost ~$500CND about a year ago. Again, our long distance costs are pennies per minute, each phone is a separate extension, we can make multiple outgoing or incoming calls, late-night callers are prompted by a menu first before the phones ring, we can control our own caller-ID, etc.

(+) DRM You can now buy the Hauppauge HD-PVR which takes the component analog outputs and converts it to a digital USB H.264 MPEG4 stream that works great with MythTV. "5C" encrypted channels can be recorded.For HD tuners using firewire and 5C flagged channels, there is no current work around (as of Feb. 21st, 2007).

Outstanding Issues

Not directly related to MythTV but, every couple of months, mdadm kicks all the HDs out of the RAID 5 array on System B. Thankfully, a reboot brings the array right back up, but very strange. This was a hardware issue, which after the MOBO upgrade, is fixed.

Every once in a while, the front end will crash, and it seems that a reset of all the MythTV processes is required. I have created a script, which when a "power" button is pressed on the remote, resets the local master backend and frontend and through a SSH tunnel it opens on the fly, resets the remote MythTV backend process (that took a bit of debugging to get a remote user to reset a different user's process) Although likely unrelated, the samba/netbios daemon stops working too.