First, when trying to start from files already on my pc, is there any way to select multiple files? It seems that when in the add file dialog only one file can be selected at a time. It would be nice if the list could just automatically populate the same as when the playlist is downloaded from the tivo. I have a bunch that i need to convert and the thought of selecting them one by one is really daunting! Also, if the file is on the pc already, is all of the metadata already gone? Selecting create metadata doesn't seem to do anything if starting from a tivo file on the pc.

Second, is there any way to put the episode number of a show in the title? most of the time with tv shows i like to name it something like: The_Simpsons_01x21.avi, so that the shows are sorted in order of original air date instead of strictly alphabetical order.

Third, is there any way you could initiate two processes at once? Specifically i think it would be very helpful to be downloading the next file to be processed from the tivo while encoding the current file.

Fourthly, is there any way to have the script download new shows as they finish recording, more like tivo desktop? Or at least to be able to easily select just the shows that have not been downloaded already?

First, when trying to start from files already on my pc, is there any way to select multiple files?

I looked into that when I 1st implemented the browser. Using the native windows file browser I didn't find a way of enabling multiple files at a time. Probably what could easily be done is an "add all" button that lets you load all .mpg/.mpeg/.TiVo files in a given directory you choose with a browser.

Quote:

Also, if the file is on the pc already, is all of the metadata already gone? Selecting create metadata doesn't seem to do anything if starting from a tivo file on the pc.

That's right, currently I get most of the metadata information from the TiVoVideoDetails url off the Tivos, not based on .TiVo files. There is some metadata in the .TiVo file itself, and tools such as pyTivoMetaGen exist that can extract it, but the data is more limited compared to what can be pulled from the Tivos. I haven't looked into seeing how hard it would be to do via Perl scripting but my guess is not terribly easy. The tivodecode binary has a means of dumping some of the metadata information so that may be the easiest route to generate from .TiVo files. If you have already converted to .mpg however, the metadata information is already gone.

Quote:

Second, is there any way to put the episode number of a show in the title? most of the time with tv shows i like to name it something like: The_Simpsons_01x21.avi, so that the shows are sorted in order of original air date instead of strictly alphabetical order.

No, but I could easily add that as a keyword, bearing in mind that some shows won't have episode #s so the keyword would get replace with nothing.

Quote:

Third, is there any way you could initiate two processes at once? Specifically i think it would be very helpful to be downloading the next file to be processed from the tivo while encoding the current file.

It certainly is possible though it adds to the complexity a little bit, but since downloads do take a long time and as long as I restrict to 1 download per Tivo at a time it would be a nice thing to do - I'll look into it.

Quote:

Fourthly, is there any way to have the script download new shows as they finish recording, more like tivo desktop? Or at least to be able to easily select just the shows that have not been downloaded already?

I've been looking into auto download capability a lot recently and even have a prototype, but hitting into some snags since it involves having to use Windows Services with Perl scripts which I'm finding out is not an easy thing to do and some things don't run properly as a Service even though they run fine as a batch script (permissions problems). So I'm not sure at this point if I can even make it work as a service. Not sure if a plain old batch mode script is useful to anyone...

I've been looking into auto download capability a lot recently and even have a prototype, but hitting into some snags since it involves having to use Windows Services with Perl scripts which I'm finding out is not an easy thing to do and some things don't run properly as a Service even though they run fine as a batch script (permissions problems). So I'm not sure at this point if I can even make it work as a service. Not sure if a plain old batch mode script is useful to anyone...

While a service would be ideal, a batch mode script would be a big step in the right direction. I could then just schedule it to run with the windows task scheduler. I typically have at least one machine on the network that is logged in.

Been using this quite a bit, man it sure helps speed up the tedious process of getting shows off the tivo and converted to xvid!

Couple of things though, it would be nice to have the option of deleting different intermediate files, like the cut mpg and mls edl and original converted mpg files, just leaving the txt and final avi files.

Also, it would be nice to have the txt file named correctly for the final format of the file, the way it is now i have to go back and rename the xxx.mpg.txt file to xxx.avi.txt.

Thanks again! I think i'm going to have to learn pearl to start messing around with this myself, too fun!

Just to let you know that I am working towards a new release, hopefully in a few more days. There are so many changes now that I will probably release as a "experimental" release, leaving the 0.3e version for download as the stable version. Some changes already in place:
* For windows, file browser can now select multiple files
* episode number is now available as a keyword to build file names
* auto downloads capability (implemented as a service in windows). There is a full GUI associated with configuration of what shows to download, and they can be by title or keyword specifications. Just as with the GUI aside from download you can also enable other actions such as metadata, decrypt etc.
* The groundwork is set for allowing multiple jobs to run in parallel, but I probably won't release this capability yet in the experimental release.

just wondering how the update is coming along? Also, I was curious what tools you use to develop this app? I've been interested in learning pearl and this project might be just the motivation I've needed..

just wondering how the update is coming along? Also, I was curious what tools you use to develop this app? I've been interested in learning pearl and this project might be just the motivation I've needed..

Thanks,
Ron

Everything I mentioned above and more I'm done with the coding and testing part... still have some work to do on updating documentation however which is much less fun. Doesn't help that I've had jury duty the last few days in addition to trying to keep up with my real job. Maybe tonight or tomorrow I will be ready to release.
As far as development tools there is nothing special needed besides a text editor since perl is an interpreted language. i.e. the source code is the execution code with no compile necessary. Since it is quite a popular programming language it's pretty easy to find examples and help online and I've been using it for years at work which helps a lot. Running perl script with -w switch is a form of 'lint' which will point out any obvious things you may be doing wrong.

I love this program and have been using it without any problems at all (once I got the directory paths pointing correctly).

Is there anyway to break up a broadcast into chunks of 1 hour sections (or some predetermined time duration)?

I use the HD-DVD recording scenario (yes I am still using my HD-DVD player
for this and my library of HD-DVD videos....I often tape them to my Betamax ) would like to break up the broadcast into smaller pieces to get them to fit in the 45 minutes that can fit on a standard DVD disk when recording HD-DVD.

As promised a new version has just been released with a bunch of enhancements. I'll be especially interested in feedback if the windows kmttg service installs and runs OK. I had trouble getting it to work initially myself on my Windows XP machine at home - not exactly sure what changed at some point where it just started working. Note the point below about needing srvany executable installed if you intend to use kmttg service for automatic transfers.

Also, if you do plan to use the auto downloads capability please consult the updated documentation for more details on exactly how the windows service works and how to make changes to the startup mode if necessary.

------------------
v0.4a (05/07/2008)
------------------
**IMPORTANT** For windows users if you want to use auto downloads service
capability you will need to install the srvany folder with the srvany.exe executable
that is part of the kmttg_win32_tools.zip file.

ENHANCEMENTS:
* Auto downloads capability now enabled. There is now a GUI you can use to
setup auto downloads by title or keyword matching.
For Windows platform you can run this as a service.
kmttg can now be run in batch modes:
-a command line option => auto download daemon mode (loops after processing)
-b command line option => auto download batch mode (exits after processing)
Consult documentation for more details on this capability.
* Multiple jobs now run in parallel where possible instead of sequentially.
The program limits to 1 file download per Tivo so as not to overload Tivos.
* For windows, file browser can now select multiple files at a time
* Episode number is now available as a keyword to build file names
* Metadata generation from .TiVo files is now supported when starting from FILE mode

Awesome! Great job! I'm just starting to use it now, haven't tried out the service part yet, but will asap. I have another question, is there a way to do a two pass xvid encoding using ffmpeg and your script?

Awesome! Great job! I'm just starting to use it now, haven't tried out the service part yet, but will asap. I have another question, is there a way to do a two pass xvid encoding using ffmpeg and your script?

Not directly using ffmpeg since right now there is a single system call for encoding but multi-pass would require additional ffmpeg calls. However, you could setup a batch script which takes input and output files as arguments then in turn calls ffmpeg (multiple times) and then use that batch script in place of the usual ffmpeg.exe binary as your encoder.

also, is there a way to initiate the batch transfer while running the script normally?

I don't understand the question. Can you elaborate?
If your question is how to run auto transfers once from command line:
perl kmttg.pl -b
That will search for any matching entries, download and process the matches and then exit. If instead you use the -a switch in place of -b it will run in a loop instead of exiting (that is essentially what the kmttg service is).

I'm wondering if there is a way to basically do the batch process (mainly starting to process just the files that match that haven't already been downloaded) from within the normal gui. this way i could start the "batch" and have the gui to see the progress.

I'm trying out the batch mode now and it seems to be hiccuping on transferring from the tivo. I'm not sure yet, but I think it might be trying to initiate more than one transfer from the tivo (maybe one per entry in the maching list?) I'll check into it more as i have time.

This rocks though! I think allowing more than one process to go at a time is going to help really increase the efficiency!

I'm wondering if there is a way to basically do the batch process (mainly starting to process just the files that match that haven't already been downloaded) from within the normal gui. this way i could start the "batch" and have the gui to see the progress.

I'm trying out the batch mode now and it seems to be hiccuping on transferring from the tivo. I'm not sure yet, but I think it might be trying to initiate more than one transfer from the tivo (maybe one per entry in the maching list?) I'll check into it more as i have time.

This rocks though! I think allowing more than one process to go at a time is going to help really increase the efficiency!

Thanks!!!

Ron

Running in batch mode and GUI mode are in no way connected so you can't start something in batch mode and monitor in GUI mode. For batch mode run you can look at "auto.log" file for a record of what it's doing. In either batch or GUI mode more than 1 download at a time per Tivo is not allowed. However, if you have multiple kmttg programs running at the same time (for example a batch mode run and a GUI mode run) that's one way you can bypass that restriction, which is not recommended.

Running in batch mode and GUI mode are in no way connected so you can't start something in batch mode and monitor in GUI mode. For batch mode run you can look at "auto.log" file for a record of what it's doing. In either batch or GUI mode more than 1 download at a time per Tivo is not allowed. However, if you have multiple kmttg programs running at the same time (for example a batch mode run and a GUI mode run) that's one way you can bypass that restriction, which is not recommended.

Hmm.. for some reason the batch script does seem to be trying to initiate two (or more) downloads from the tivo at the same time.. I attached my log file so you can see. right after creating all of the metadata files it initiates two separate transfers at the same time. Also, I think it might be good to limit the number of concurrent jobs that try to run at one time.. i tried to initiate a batch transfer of all the files on my tivo and after quite a few curl instances are started error messages pop up and the rest fail to run. If you'd like i can get a log of that too. I also attached my ini file in case you needed something from there..

Hmm.. for some reason the batch script does seem to be trying to initiate two (or more) downloads from the tivo at the same time.. I attached my log file so you can see. right after creating all of the metadata files it initiates two separate transfers at the same time. Also, I think it might be good to limit the number of concurrent jobs that try to run at one time.. i tried to initiate a batch transfer of all the files on my tivo and after quite a few curl instances are started error messages pop up and the rest fail to run. If you'd like i can get a log of that too. I also attached my ini file in case you needed something from there..

Thanks again, this is awesome!

Ron

Thanks for the logs, that was helpful. Yes it looks like when jobs are queued up quickly such as in the batch case more than 1 download per Tivo is getting through and now I can see how by looking at the code. I will have to fix that. Also, I will need to restrict metadata generation to 1 Tivo at a time as well since they are grabbing data from Tivos as well.

v0.4b just released.
------------------
v0.4b (05/09/2008)
------------------
BUG FIX:
* More than 1 download at a time per Tivo was being allowed through with
previous version. This has been corrected.

v0.4b just released.
------------------
v0.4b (05/09/2008)
------------------
BUG FIX:
* More than 1 download at a time per Tivo was being allowed through with
previous version. This has been corrected.

So far so good! I'm also trying out a batch file, as you suggested, to do a two pass xvid encode.. working on the first pass now.

Thanks a whole lot for going through all the effort to make this great utility set; it automates a lot of what I wanted to do.

I gotta beg though... Can we set up a cleanup script that allows me to clean up the files after it has finished processing? And also, is there a way to have it delete the file on the Tivo after downloading?

I've installed it on my Windows Home Server, and along with pyTivo, it now completes my home media server toolset.

The service appears to intall and run fine on WHS (essentially windows server 2003) and all of the 3rd party tools run fine too.

The file cleanup options are very nice, but one more request would truly make it a "be all and end all" for me.

Can you add the option to have a user-defined application command line launched at the end of the other jobs for a given download?

I would like to have a batch file (or powershell script eventually) that I can run after the other jobs for a download so I can move the program to the shares that pyTivo watches, or arbitrarily do other stuff. The extensibility would allow me to do other things eventually as well as I continue to integrate into home server.

Thanks again for this tool. Tivo Desktop does not play nicely on WHS, and this now provides the critical missing piece - which was the autodownload and decrypt. The COMSKIP/COMCUT are gravy and I need to play with them determine whether they will be useful for me.

Can you add the option to have a user-defined application command line launched at the end of the other jobs for a given download?

I would like to have a batch file (or powershell script eventually) that I can run after the other jobs for a download so I can move the program to the shares that pyTivo watches, or arbitrarily do other stuff. The extensibility would allow me to do other things eventually as well as I continue to integrate into home server.

Note that you could define the "encoder" executable as a script instead of a normal encoder executable to do whatever you wish. For "encoder_args" you can include INPUT keyword so that the .mpg file is passed to your script and then in the script you can do whatever you like. If you are encoding you can embed the encoding within your script and then do the post-processing. The encoder step is the last step kmttg calls which makes it well suited to the task of doing your own custom post-processing.

Note that you could define the "encoder" executable as a script instead of a normal encoder executable to do whatever you wish. For "encoder_args" you can include INPUT keyword so that the .mpg file is passed to your script and then in the script you can do whatever you like. If you are encoding you can embed the encoding within your script and then do the post-processing. The encoder step is the last step kmttg calls which makes it well suited to the task of doing your own custom post-processing.

Aah... couldn't see the forest for the trees on that one.

That will work perfectly. I don't use the encode job currently since I am just using kmttg to download off of the Tivo so that I free up the internal hard drive space on the Tivo for recording new stuff.

That will work perfectly. I don't use the encode job currently since I am just using kmttg to download off of the Tivo so that I free up the internal hard drive space on the Tivo for recording new stuff.

Great tip and thanks again.

I'm off to brush up my batch file skills...

Ted

If you already tried it you probably noticed... You'll get an error by kmttg once your script completes since you are not actually encoding to a file and kmttg will check if file exists. However it shouldn't interfere with anything other than that error message.

As promised a new version has just been released with a bunch of enhancements. I'll be especially interested in feedback if the windows kmttg service installs and runs OK. I had trouble getting it to work initially myself on my Windows XP machine at home - not exactly sure what changed at some point where it just started working. Note the point below about needing srvany executable installed if you intend to use kmttg service for automatic transfers.

Also, if you do plan to use the auto downloads capability please consult the updated documentation for more details on exactly how the windows service works and how to make changes to the startup mode if necessary.

------------------
v0.4a (05/07/2008)
------------------
**IMPORTANT** For windows users if you want to use auto downloads service
capability you will need to install the srvany folder with the srvany.exe executable
that is part of the kmttg_win32_tools.zip file.

ENHANCEMENTS:
* Auto downloads capability now enabled. There is now a GUI you can use to
setup auto downloads by title or keyword matching.
For Windows platform you can run this as a service.
kmttg can now be run in batch modes:
-a command line option => auto download daemon mode (loops after processing)
-b command line option => auto download batch mode (exits after processing)
Consult documentation for more details on this capability.
* Multiple jobs now run in parallel where possible instead of sequentially.
The program limits to 1 file download per Tivo so as not to overload Tivos.
* For windows, file browser can now select multiple files at a time
* Episode number is now available as a keyword to build file names
* Metadata generation from .TiVo files is now supported when starting from FILE mode

I'm really loving this! Still haven't tried out as a service yet, still tweeking my 2 pass batch file, but i really love it running more than one process in parallel.

Came across another gotcha though... If you select a bunch of mpg files or I would guess tivo files and initiate encoding it fires up processes for however many you select. This quickly brings my system to a screeching halt.

I think it would be best to limit the maximum number of processes that can run at once.. either by type of job or just the total. It would be awesome if that limit could be part of the config file so that if someone has 8 cores they can choose to run 7 or 8 or whatever number, and if someone only has one or two they could set the number lower.

Also, it would be great if when the kmttg script is closed it automatically terminated all of the threads it started up, otherwise i have to go through and close them in task manager. I know you can cancel them through the gui, but in the case when 53 instances of ffmpeg have been started the gui just doesn't work at all. Basically my only choice then is to log-off or reset the system.

Also, i'd like to second the request for an optional final "clean up" batch file. Just another line in the config area that you can enter the path to a file that is called after the encoder process along with an args line like with the encoder, that way it can be called even if you don't do the encoding step. Also, if there was an additional variable that you could pass to the script that indicated what steps had been preformed on the file (create metadata, decrypt tivo, comskip, ecode) the called program or script could be more intelligent about it's cleanup. In other words we would be able to add say OPTIONS to the args for the cleanup script in the same way we can pass INPUT and OUTPUT.

Make any sense?

Oh ya, one other thing that I know would help me, if there was a way to automatically add all of the existing files (or files selected in the gui would be even better) on the tivo to the history file so that if they're already processed they won't have to be processed again in the batch/service mode. And along with that it would be helpful to have the title or file name or something along with the program id in the history file, so that we can look at the file and see what files are marked as processed already.

New version just released:
------------------
v0.4d (05/12/2008)
------------------
ENHANCEMENTS:
* Added "active job limit" setting to configuration which limits the number of CPU
intensive jobs that can run at the same time to this number (defaults to 2).