I decided to start this thread because while there are not a huge number of people running applications such as Galleon, pyTivo, or HME for Python under Linux, there are still some of us, and perhaps a few lurking in the bushes who would consider dipping their toe into building a Linux server because of the speed, simplicity, expanded feature set, and stability of a Linux server. Others, perhaps may be thinking of hacking their Linux-based NAS boxes so they can run one of the aforementioned apps.

Basically, anything related to Linux and a networked TiVo is open for discussion, here. Anything related to specific features of one of the popular HMO or HME applications is probably better suited to one of the application specific threads elsewhere in the forum, but if your need is to write a script for handling downloaded videos or for starting pyTivo automatically at boot time, running an application under wine, or even getting a greenfield Linux installation set up for use with a TiVo, this is the place to be.

There are several members of this forum who have extensive experience with *nix based systems, and I invite them to participate. I am sure they will be happy to offer assistance to TiVo owners with limited (or no) Linux experience. While I am no Linux expert, I do have some experience with the OS and with interfacing it to TiVo DVRs, and I will be happy to offer my own help as far as I am able.

So, anyone who is interested or needs help with any aspect of Linux and TiVo, speak up!

I'm sure I'll be a frequent supplicant. In time, perhaps a resource as well.

Yep. That's the idea. I have to say it really is satisfying to hear of people who benefit from my suggestions, and I think many others are the same. It's not unlikely that one day your expertise may well exceed mine, and that will be just fine by me.

I think what would really help people using linux are installation scripts for pyTiVo and that do all the grunt work for you. I'm running an old version of the program and have resisted upgrading simply because I don't want to have to spend an hour or two reconfiguring the interface. As it stands now I have only the media server part working and most of the time in order to recognize new shows on the server I have to restart the service.

All of this might have been addressed in later versions, but I haven't seen a place that details what's now involved in the installation or configuration of pyTivo.

Good thread though, I'll be following it.

__________________
137hr DTS2
20/180hr HD
166hr DTS2
180hr S2

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

I don't know of any that have been puslished, but then installation under Linux is distro dependant and also highly personal. For pyTiVo, HME for Python, and vidmgr, it's pretty trivial though, except for the dependance based init scripts, if any. I have a couple of update scripts that I use, and if you will give me a couple of days or so, I can spiff them up for general distribution. Perhpaps someone else will contribute, as well.

Note the CONFIGURATION of pyTivo, etc., is beyond the scope of this thread, and is handled by other, platform independant utilities. Installation under Linux is another matter, however, and fits very nicely in this thread.

OK, here is the first script. It is designed to upgrade to the newest version of vidmgr, whenever Jeff may decide to release one. It could also be used to partially install vidmgr the first time. Using this script requires the following:

1. HME for Python must already be installed in the <Target> directory.
2. At least one version of vidmgr must be downloaded into the <Source> directory.
3. While in the <Source> directory, issue the command

Code:

tar -xzvf jbernardis-pytivo-video-mgr2-XXXXXXX.tar.gz

where XXXXXXX is the version number of the downloaded revision of vidmgr.

4. In the following script, update the value of SourceDir to <Source>
5. Update the value of TargetDir to <Target>/vidmgr
6. Run the script.

This creates a file in /var/run named pyHME.pid that contains the PID of the running HME for Python process and a log file named pyhme.log in /var/log containg HME for Python's stdout stream. It puts the program into the background and divorces it from the parent script, allowing the app to run even after the terminal which spawned it is gone. It's parent process then becomes PID 1, the system init app.

To automatically run HME for Python at startup on a Debian based system employing dependency based booting, one can create a file such as the following in /etc/init.d.

Once again, simply download the git tarball into <Source>, and untar the file. William uses a slightly different directory format for his git, so if you are using his fork, you should have:

Code:

AppName="wmcbine-pytivo-*"

The script, when run, will find all directories under <Source> that match the string $AppName, select the newest, and copy all the files in that directory to <Target>.

In the gits, the configuration files are named XXXX.dist. That way, when the script runs, the default configuration should not overwrite your custom settings. The first time the software is installed, the user should run the command

The following only applies if the user is employing dependency based booting on a Debian or Debian derivative system. If one is employing legacy (manual) boot script ordering, it does not apply, and of course it does not apply to any distro that is not Debian based, although the scripts themselves should work with just about any distro.

Notice in the LSB header of the /etc/init.d/pyHME script there are two lines:

This tells insserv that the script in question should not be run until after the "pyTivo" service is up and running. I don't actually know that this is strictly necessary in this case, but there might be some unexpected results if one tries to run vidmgr or jukebox before pyTivo, upon which they depend, is active. It also makes sure HME for Python is shut down prior to shutting down pyTivo when the sytstem is shutting down or rebooting. I'm pretty sure this is not necessary in this case, but it is a good practice in general.

In order for this to work properly, one must register pyTivo as a service. To do so, one simply edits the file /etc/insserv.conf and adds the line

Code:

$pyTivo pyTivo

This tells insserv the script /etc/init.d/pyTivo implements the pyTivo service. When ordering the boot scripts, then, update-rc.d and insserv make sure the /etc/init.d/pyHME script is run after the pyTivo script and is shut down prior to pyTivo being shut down.

Both pyTivo and the TiVos themselves have web interfaces. Consequently, as most of you know, a web browser can be used to control them. In particular, pyTivo has a web interface that not only allows one to configure pyTivo, but also to push videos from the server to a TiVo or to download videos from a TiVo to the server. Similarly, the TiVos themselves allow downloading of videos to the host machine via secure HTTP using a browser. There is a utility out there, however, that allows a person writing a script (or an app like kmttg, as a matter of fact) to automate web operations. That utility is curl. Strictly speaking, curl is not a Linux app, because there are ports for other Operating Systems, including DOS/Windows. I mention it here, however, because there is a Linux port, and I think those of us running Linux servers may well want to implement curl to handle various aspects of Home Media.

In short, curl will take a CL argument and pass it to any web server. This means essentially anything that can be done using a browser can also be done using curl. For example, I wrote a web application that itself uses curl to control pyTivo. From anywhere in the world, I can log in to my sever and view almost all of the videos on the server. From there I can select one and have pyTivo push the video to one of my TiVos. The command used to implement this is:

If one redirects the stdout and stderr of pyTivo to a log file, after a while that log file can grow quite large and unweildly, as is the tendency with log files. Many Linux distros include the logrotate utility for this very reason.

When logrotate runs, it checks its configuration directory (typically /etc/logrotate.d) for target applications to manage. Each managed application will have its own configuration file. In this case, one might create a file named "pyTivo", with its configuration options. Given the nature of pyTivo's output and the nature of what it does, it is probably sufficient to keep no more than 1 - 2 weeks of logs, and it is probably not necessary to compress the logs. Due to the way logrotate works, this means deleting the previously saved log file, renaming the current log file, and creating a new, empty log file.

Because logging of pyTivo is accomplished by redirecting stderr and stdout, the application does not open and close the log file every time it needs to write, but rather the file is opened and kept open until the application is terminated. Thus, even if an external process like logrotate renames the log file (or even deletes it), the application continues to write to the same file. Thus, the old log file will continue to grow, even though it has a new name (or even has been deleted), and nothing will ever be written to the new log file. This is good in that even if the application is currently writing to the log file, renaming the file doesn't interrupt or corrupt the logging. It is bad, of course, in that logrotate hasn't really done anything very useful at this point.

In such cases, it is necessary to stop and restart the application so that it will stop writing to the old log file and start writing to the new one. Well, that is simple enough.

First of all, logrotate provides a directive for just such a purpose. It is called "postrotate". When logrotate encounters this directive and its terminator, which is named "endscript", it executes all the commands between the two markers sequentially. One could then just put something like this in the config file:

Code:

postrotate
/etc/init.d/pyTivo restart
endscript

This will certainly work, but there is a bit of a hitch.

The second thing is that using this elementary approach is liable to kill pyTivo while it is right in the middle of sending a show from the server to a TiVo. That wouldn't be very nice, although depending on one's viewing habits it probably would not happen all that often. One way around this would be to send pyTivo a signal that tells it to shut down gracefully after it is done transferring all its current jobs. I do not know if William has implemented or would be interested in implementing a trap in his code to handle this sort of signal, however. If he reads this and responds in the affirmative, then there is a fairly slick way to handle the situation. Not knowing if that is or ever will be the case, then, we have to handle it in a little more of a brute force fashion. The following script will check to see if pyTivo is transferring any files, and if so wait 60 seconds to check again and do so continuously until it isn't transferring anything, and then restarts the application.

Edit 11/06/2012: I ran across what is effectively a bug in the following routine this morning. If one of the transfer processes hangs (or one runs Jukebox, turns off the TV, and forgets about it), the script below could loop forever. I've added a timer that kills the process aftrer 6 hours. If that isn't long enough, increase the value of the Trans_Elapse variable.

Of course, the $VideoDir, $PIDFile, and $InitFile variables would need to be adjusted to match the individual parameters of the user's system. With the above script in place in /usr/share/pyTivo/logcheck, the logrotate config file could be:

Now, there are a couple of vulnerabilities to this approach I need to mention. The first is this creates a bit of a race condition, where between the time the script decides pyTivo isn't busy and the time the init script kills the current instance of pyTivo, pyTivo could possibly start transferring a video. Given this period of time is at most a few milliseconds, it's quite unlikely, but it is possible. Perhaps the most likely thing is a push to one of the TiVos is scheduled, but has not yet actually started to transfer when pyTivo is killed, and the Tivo then looks for a non-existent pyTivo server. The good news is this would be an exceedingly rare occurrence, and it would only happen at the very beginning of a transfer, not half-way through or almost at the end.

The other vulnerability has to do with the user creating multiple shares on a system. The $VideoDir variable points to the highest directory where all the shares are found. Thus, in my system, all the shares are subdirectories of the /RAID mount point. I have /RAID/Recordings, /RAID/Music, /RAID/DVD, etc. All of the files are either somewhere in /RAID/xxx[/yyy/zzz...] themselves or are symlinks pointing to files in /RAID/xxx[/yyy/zzz...]. Thus, although the `lsof` utility finds a fair number of open files throughout the system (see below), the filter `grep -q "VideoDir"` insures that only files somewhere in /RAID are considered to be ones being transferred by pyTivo. If the user has multiple pyTivo shares configured and any of them are not in the subdirectory tree of a single parent directory, then those files will not be noticed by the script. For example, if my share were not /RAID/Recordings, /RAID/Music, /RAID/DVD, etc. but rather /Recordings, /Music, /DVD, etc, then no single value of $VideoDir will suffice to catch all the shares except for "/". The problem with using "/" as the filter, however, is while it will without question produce a positive result if any file is being transferred, it will in fact always produce a positive result because pyTivo always has a number of files open that are not transferring videos:

The first script, contained in the directory pointed to by the $DAEMON variable in the second script, is what actually runs and daemonizes pyTivo, setting up the log file and the PID file, spawing pyTivo as a detached process in the background, and returning a value that reflects whether this process was successful or not. It can be run by itself to start pyTivo, provided of course pyTivo is not already running. This file can easily be modified to meet changing situations specific to pyTivo.

The second script is the init script, which should be much more static than the pyTivo script, merely calling the daemon script and putting the result into the startup logs.

Doing it this way makes things more modular and more future-proof, allowing for there to be changes in the particulars of the pyTivo daemon without having to monkey with the init script while also allowing for there to be future changes to insserv or the init process without having to deal with any particulars of the pyTivo daemon. It also can make trouble-shooting one or the other much easier.

Ok, I made the first script a file in the pyTivo directory then altered the second script to point to that file. It works but I still get the same results, I have 9 in my movies folder until I restart pyTivo.

I'm can't even figure out what's common about the 9 that show up but it's consistently the same 9 shows???

Ok, I made the first script a file in the pyTivo directory then altered the second script to point to that file. It works but I still get the same results, I have 9 in my movies folder until I restart pyTivo.

I'm can't even figure out what's common about the 9 that show up but it's consistently the same 9 shows???

It's been a while, so I'm fuzzy on the details, but I seem to recall something similar happening to me in the past. I'm not certain, but I think it had to do with zero length files, or maybe broken symlinks. Are any of the video files or metafiles in any of your shares zero length or otherwise corrupted?

Another guess would be files with problematical file names. Do any of the file names (including directory names) on any of your shares have a name with any of the following characters:

: \ ' " ` ? & /

A leading period ( "." ) could also cause trouble.

Symlinks to non-existent files can also be trouble. (In fact, IIRC, I think that is what caused my trouble.) Do an `ls -l` in all of your shares to make absolutely certain there are no symlinks and if there are, make absolutely certain they are not broken.

Also, you are not specific whether this is in the NPL or under vidmgr. Are you running vidmgr? If so, are the results there the same as in the NPL? If you aren't running vidmgr, you might consider running it if for no other reason than troubleshooting this issue. Also, it occurs to me it might help with diagnosis if you run pyTivo in debug mode. Set "debug = True" in the config file, restart the server, and try to duplicate the issue. Look at the log file, and perhaps reproduce it here.

What is causing this resolution error and how do I fix it? The server keeps running and togo will work but the Galleon icon disappears from the HDtivo.

This probably belongs in one of the old Galleon threads and not this one, but ...

I don't think the resolution line is necessarily the error. What version of Galleon are you using? You could also try to turn on debugging so there's more information in the log that might help determine the cause of that "event opcode 10."

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

I recently install Galleon on my kubuntu 12.04 system, it has been several years since I originally installed it but want to use Galleon so my wife can monitor the traffic cams during my drive home and alert me if the roads are closed.

Hey, that's kinda cool.

It would be easier to read your logs if you did not use a tiny font and wrapped them in code tags:

I'm about to take the TiVo plunge now that I can get Comcast's OnDemand on TiVo (yes!) and my 2nd Comcast DVR just crashed. I've been using BeyondTV for a long time recording off SD cable boxes while using Comcast's box for HD recordings, so I'm really interested in moving files between TiVo and PC. I think it's really cool that there's all this 3rd party open-source software for TiVo, but I need some help understanding what everything does.

Can someone please put together a post with a short description of the currently available programs including:
program name
what does it do
where to download it

It would be really nice if this could be put into a closed sticky thread so newbies like me could easily find it.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

I'm about to take the TiVo plunge now that I can get Comcast's OnDemand on TiVo (yes!) and my 2nd Comcast DVR just crashed. I've been using BeyondTV for a long time recording off SD cable boxes while using Comcast's box for HD recordings, so I'm really interested in moving files between TiVo and PC.

This topic is not really suited to this thread. It would be best moved elsewhere. If you have questions or comments on getting any of the applications you seek working under Linux, then come on back here.

Note I do not believe you will be able to move any OnDemand content to a PC.

Quote:

Originally Posted by JeffDwork

Can someone please put together a post with a short description of the currently available programs including:
program name
what does it do
where to download it

As you can see from the link provided by windracer, there are something over 100 3rd party applications available for the Tivo, and some of it has quite an extensive array of features. Writing up a "what does it do" synopsis for every one of them would be quite an undertaking for any one individual - especially an unpaid one.

Quote:

Originally Posted by JeffDwork

It would be really nice if this could be put into a closed sticky thread so newbies like me could easily find it.

Um, why a closed thread? New apps for TiVos come out on a regular basis. Some of the very best have been developed within the last few months. Any closed thread of this nature would soon be obsolete, sticky or not.

Unfortunately, while that thread is close to being comprehensive in terms of the names and locations of apps, it is not so when it comes to functional descriptions. I'm afraid JeffDwork is just going to have to do some research, but that thread is an excellent place to start.

Have you checked your system logs for an event right about the same time?

The log you posted looks to me like input to log.txt. Have you checked wrapper.log?

Thanks again, I think the resolution errors may have been associated with the video setting on the HD Tivo after I switched to 1080i hybrid it seemed to work better.

I am now using a slightly overclocked Raspberry Pi (900mhz) running the 32 bit 2.3.1 version of Galleon for the traffic cams. The Raspberry Pi is slower but once the traffic cam is up it seems pretty good. I added a call to galleon/bin/run.sh to the /etc/rc.local so if my wife has any problems she can just switch the Raspberry Pi off and back on again. If and when I want to use xbmc I can just switch SD cards and reboot. The only thing setup in Galleon is the Internet Images, and I do not plan on using it for anything more at this time.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

The traffic cams via Raspberry Pi with Galleon has been working very well for a couple days. I have ordered a second $35.00 Raspberry Pi to use with an old, usb wifi adapter & usb cam currently gathering dust in hopes to send my own cam images. It may be possible to use the wireless webcam setup to do video and later push it to the Tivo but I need to do more research, I read someone had motion working with the Raspberry Pi but have a lot to learn.

It's ironic, a few years ago we used Pytivo a lot but after getting Roku and a HD media player we stopped using it, but if I can get my wireless usb cam video working with motion might install pytivo again and try pushing home made videos to our Tivo, it will be a nice addition for our current surveillance/dvrs setup and having an additional capabilty of security surveillance videos would be much more useful to me than many of the features on Tivo these days. It's a long way off, now just an idea, but if I could get security videos on the Tivo in folders by date it would be really nice addition to the Tivo for me. Most stand alone dvr systems for security surveillance only work with windows, not much for linux unless you do it yourself.

Thanks for the suggestions, and you script examples. I have used pytivo & and also curl in the past to link a ftp directory on my media player to my Kubuntu file system. I will take a long look at it all, may just use the HD Media player for the new video setup because it will allow me to zoom in on areas of the video, which will be better for security videos. Either way I prefer view them on my big screen TV rather than a 24" monitor. Galleon still seems to be working OK and so far appears to be a unique program that must be used on Tivo.

I am currently using WHS and streambaby. I had a drive go bad and it was a real pain to save and move my data and videos. I am thinking of going to a new setup with either a Netgear or Synology server. I am thinking the netgear / sinology setup might be easier to maintain, ie hot swappable. Netgear has some Tivo support, but in some ways Synology seems better. Can anyone give me an idea which is better and how hard would it be to set up either one with PyTivo so I can stream videos from the server to my Tivos. Any input would be appreciated.

I have installed Windows thousands of times , built all of my computers and the WHS. I have NO experience with Linux.