If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

TyExtract - Windows command line MFS_FTP batch extraction utility

Attached is a Windows command line MFS_FTP batch extraction utility that I wrote in order to automate the process of extracting multiple shows from your Tivo/Tivos via a single process, and I thought that I would share it with the group.

The utility is intended to be run from the Windows Scheduler, so that on a specified interval, it will automatically run and extract programs that contain your specified "keyword(s)" in their filenames, which have been recorded within a specific day range.

In this way, rather than having to connect to my Tivo to see what has been recorded, select the files, and then wait for them to download before I can then begin editing them, I will have my shows waiting for me, so that I can begin editing right away.

I built this using the WinInet libraries, and as such, your PC will require Internet Explorer 4.0 or greater, since the required DLLs accompany this application when installing, and obvioulsy, since TyExtract is an MFS_FTP command line utility, you need to be running the MFS_FTP server on your Tivo.

The attached zip file contains the executable, tyExtract.exe, a README.TXT file with detailed usage instructions, an example .bat file setup (example.bat), and an example of the resultant messages that get generated when the utility is run (tyExtract.log).

Can you tweak it to allow downloading by "thumbs up" rating in addition to the date.

I'm limited to the information that is supplied to me via the MFS_FTP server, in terms of the 'show name' and 'file create date'. If MFS_FTP could be modified to include, as part of the 'show name' string, a thumbs rating value, then it could be done. However, this is not currently part of the standard 'show name' string that does get generated.

would you consider allowing wildcards .. as in -s * so that it would transfer anything matching by date range switch. This way it would get everything, recorded.

can't wait to try it out!

Just posted a new version (v1.2) in the .zip file above, that has added wildcard support, i.e. '-s *', so that you can download ALL recordings made within the date range specified via the '-d' flag, regardless of their show name.

Also added automatic logging using the '-g' flag, so that screen messages will be automatically written/appended to tyExtract.log. This file will be located in the same directory that you have specified for your recordings to be saved, using the '-l' switch.

Just posted a new version (v1.2) in the .zip file above, that has added wildcard support, i.e. '-s *', so that you can download ALL recordings made within the date range specified via the '-d' flag, regardless of their show name.

Also added automatic logging using the '-g' flag, so that screen messages will be automatically written/appended to tyExtract.log. This file will be located in
the same directory that you have specified for your recordings to be saved, using the '-l' switch.

having trouble figuring out the 'day:day' switch. It seems to be off by 3 days... tried a few times with 0:0 0:6 finally after 3:3 it seemed to think that that was recordings for today. ??

here's a copy of the console results as the log doesn't show the commandline.

Code:

G:\_downloads\_TIVO\TyExtract\tyExtract.12>tyExtract.exe -h 192.168.1.111 -s * -
l d:\_tivo -d 3:3 -v -g
Running in verbose mode.
Attempting to establish connection on 05/19/04 at 19:46:35...
Connection established with '192.168.1.111' on port 3105
Matching Show: The Tonight Show With Jay Leno-2004.04.29-No Title.ty
Recorded on: Thu, April 29
Request range: Wed, May 19 to Wed, May 19
Sorry, recording does not fall within range
Matching Show: The Tonight Show With Jay Leno-2004.04.30-No Title.ty
Recorded on: Fri, April 30
Request range: Wed, May 19 to Wed, May 19
Sorry, recording does not fall within range
Matching Show: The Pathfinder-.15 AM Wed May 19, 2004-No Title.ty
Recorded on: Wed, May 19
Request range: Wed, May 19 to Wed, May 19
Show is within requested range
Downloading 2.1 GB

I also tried to get recordings with SHO hoping to get recordings from Showtime .. the tmf files have {SHO} in the file name can I put that complete string in? including the {} characters? are there limits or "special" character restrictions?

having trouble figuring out the 'day:day' switch. It seems to be off by 3 days... tried a few times with 0:0 0:6 finally after 3:3 it seemed to think that that was recordings for today. ??

here's a copy of the console results as the log doesn't show the commandline.

Code:

G:\_downloads\_TIVO\TyExtract\tyExtract.12>tyExtract.exe -h 192.168.1.111 -s * -
l d:\_tivo -d 3:3 -v -g
Running in verbose mode.
Attempting to establish connection on 05/19/04 at 19:46:35...
Connection established with '192.168.1.111' on port 3105
Matching Show: The Tonight Show With Jay Leno-2004.04.29-No Title.ty
Recorded on: Thu, April 29
Request range: Wed, May 19 to Wed, May 19
Sorry, recording does not fall within range
Matching Show: The Tonight Show With Jay Leno-2004.04.30-No Title.ty
Recorded on: Fri, April 30
Request range: Wed, May 19 to Wed, May 19
Sorry, recording does not fall within range
Matching Show: The Pathfinder-.15 AM Wed May 19, 2004-No Title.ty
Recorded on: Wed, May 19
Request range: Wed, May 19 to Wed, May 19
Show is within requested range
Downloading 2.1 GB

Thanks for the feedback.

Are the "Recorded On" dates displaying correctly for your shows, and just not the request range?

If so, what Timezone are you in? Normalizing the Tivo filesystem time with your current localtime can be tricky, and I am doing some timezone offsets, etc, in an attempt to compare apples to apples. Also, since I am using your system time, make sure your date and time on your PC is accurate.

I'll get to the bottom of it once I get a bit more feedback. Interesting that it appears to be off by exactly three days. You are correct in that 0:0 should be equivalent to today. If you don't enter anything for the '-d' flag, which is equivalent to 0:0, does the "Request Range" actually show as 3 days into the future?

also tried to get the "tmf" instead of the "ty" and it still search for the "ty" files...

That's just a bonehead move on my part. It actually IS downloading the "tmf" file. In attempting to "clean" the file name to something more condensed, I inadvertently hardcoded the extension to .ty, regardless of the "type" specified. The actual file is right, just not the extension ;-(

I'm going to remove the "cleaning" of the filenames all together for now, in order to get a revised version up ASAP. This will correct the extension naming error, and will write the output file exactly as it is provided by the MFS_FTP server. While verbose, at least it will be correct.

As far as what you can pass in via the "-s", show name flag, yes you can include the open and close braces, etc, in order to get more finite. You can essentially include any and everything that is actually part of the entire show name string, to the point that if you entered it exactly as it appears in your FTP client, you would end up making a request to download a single file. Just make sure to surround your name in double quotes if it includes any spaces, i.e:

Are the "Recorded On" dates displaying correctly for your shows, and just not the request range?

If so, what Timezone are you in? Normalizing the Tivo filesystem time with your current localtime can be tricky, and I am doing some timezone offsets, etc, in an attempt to compare apples to apples. Also, since I am using your system time, make sure your date and time on your PC is accurate.

I'll get to the bottom of it once I get a bit more feedback. Interesting that it appears to be off by exactly three days. You are correct in that 0:0 should be equivalent to today. If you don't enter anything for the '-d' flag, which is equivalent to 0:0, does the "Request Range" actually show as 3 days into the future?

I'm in EST (GMT -5:00) and my PC is in sync via timeserver -

and yes the recorded on time is acurate, if you notice that the last recordind matches and is downloading ...

and here is a dump I just made .. with -d 0:0 show the future dates...

Code:

Running in verbose mode.
Attempting to establish connection on 05/19/04 at 22:33:17...
Connection established with '192.168.1.111' on port 3105
Matching Show: The Pathfinder-.15 AM Wed May 19, 2004-No Title.ty
Recorded on: Wed, May 19
Request range: Sat, May 22 to Sat, May 22
Sorry, recording does not fall within range
Matching Show: The Ellen DeGeneres Show-5.19-No Title.ty
Recorded on: Wed, May 19
Request range: Sat, May 22 to Sat, May 22
Sorry, recording does not fall within range
*** No shows with '*' found in your Now Playing List,
recorded between Sat, May 22 and Sat, May 22. ***

Okay. I'll have to dig into my time calculations to see what is happening. I've only to date used the program on my machine, which is CST, and haven't had any problems, so I'll have start doing some debugging.

In the meantime, I've posted another update, v1.2.1, to fix the file naming convention error. Please give it a try with the various extensions for the "-t" flag and let me know if you now get what you expect.

Again, the resultant filenames will be much larger, as they should be a direct pass-through of what MFS_FTP provides, rather than me trying to "pretty them up".

PORT switch also not working...

I have an S1 DTIVO mfs_ftp running on port 24 I set the -p 24 but would never get any recordings, even thought the log sdaid it connected. Tried serveral times and dates.. nothing. I then went in and killed MFS_FTP and restarted on the default port 3105, everything worked after that, still had the 3 day issue but it transfered those files that matched. That was the only change I made, I took out the -p 24 and restarted mfs_ftp on default ---

I have an S1 DTIVO mfs_ftp running on port 24 I set the -p 24 but would never get any recordings, even thought the log sdaid it connected. Tried serveral times and dates.. nothing. I then went in and killed MFS_FTP and restarted on the default port 3105, everything worked after that, still had the 3 day issue but it transfered those files that matched. That was the only change I made, I took out the -p 24 and restarted mfs_ftp on default ---

Check the contents of your port.24.log in your MFS_FTP directory for errors. I've got the port stuff working pretty well. I shutdown my MFS_FTP server and started it back up on port 24 to see what happens, and it fails as well, and I see the following in my log:

You can see that tyExtract is able to connect, but then is kicked out by the server with a socket issue. The same exact thing happens to me when attempting to connect using FlashFXP and smartFTP on port 24, so it appears to be an error while attempting to use this specific port (24).

Do you see the same errors in your log file?

I shutdown and ran on various other random ports (high numbers), such as 2000, 2222, 3333, and all worked perfectly. Try starting up on some of these ports, and let me know your findings.

Other items: I'm going to dig into the code today to see if I can determine what is going on with the "day range" being messed up for you. I've even reset my computer to your timezone, and still can't get the error to repeat.

Also, I've added to the top of the log file writing, the command line arguments, so that, for debugging purposes, it will first show the commands you passed in, followed by the resultant output. So in the next release, hopefully, in addition to the day range fix, this little change will also be incorporated.

Correct me if I am wrong, but the "day range" issue seems to be the last major outstanding issue, provided my last release resolved your issue with improper extension naming (always .ty), which I believe it should have.