FYI, it looks like there is a command line version available in this zip file which could be used to batch a bunch of jobs. I guess I should think about adding that to kmttg as an option for Windows users.

Thanks much for the link, I hadn't been able to find a download of DirectShow Dump. I did use your link to get DSDCmd.exe which I tried in my Windows 7 virtual machine.

The only documentation I could find was the /? switch which shows -s and -t. I used a kmttg TS .TIVO file for -s and gave a directory for -t (and then a filename I made up as MM.mpg). There was no output in either case. Possible a permissions issue though I opened up the destination disk with chmod 777 -R and then checked that it's all writeable.

Yes, you need at least a partial install of TiVo Desktop with TiVoDirectShowFilter.dll installed and registered for DirectShow Dump or VideoRedo to work. A partial install is achieved by first installing TiVoDesktop and configuring it with MAK, etc. then uninstall and choose partial uninstall to keep the components needed to decrypt .TiVo files. TiVo is hiding TiVo Desktop these days, but the link is:http://assets.tivo.com/assets/exe/ti...sktop2.8.3.exe

Then if running kmttg on Windows run Help->Update tools to download updated tools package which contains DirectShow dump.
Then you can find an option to enable under config->Program Options: "Decrypt using DirectShow Dump instead of tivodecode"

The command line for DSDCmd.exe is quite simple:
"\path\to\DSDCmd.exe" "-s:\path\to\file.TiVo" "-t:\path\to\file.mpg"
(The quotes around everything is in case your paths and/or file names have spaces).

(Note that DSDCmd.exe is not setup to accept stdin instead of a file, so combined download/decrypt is not possible).

BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.

__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.

For kmttg encodes the format is always explicitly defined, so the file extension name doesn't really matter. Having said that, when using VideoRedo for decrypt I do actually use .ts extension for TS .TiVo decrypt/qsfix instead of .mpg. (Also use .ts for H264 .TiVo decrypt).

Yes, you need at least a partial install of TiVo Desktop with TiVoDirectShowFilter.dll installed and registered for DirectShow Dump or VideoRedo to work. A partial install is achieved by first installing TiVoDesktop and configuring it with MAK, etc. then uninstall and choose partial uninstall to keep the components needed to decrypt .TiVo files. TiVo is hiding TiVo Desktop these days, but the link is:http://assets.tivo.com/assets/exe/ti...sktop2.8.3.exe

Then if running kmttg on Windows run Help->Update tools to download updated tools package which contains DirectShow dump.
Then you can find an option to enable under config->Program Options: "Decrypt using DirectShow Dump instead of tivodecode"

The command line for DSDCmd.exe is quite simple:
"\path\to\DSDCmd.exe" "-s:\path\to\file.TiVo" "-t:\path\to\file.mpg"
(The quotes around everything is in case your paths and/or file names have spaces).

(Note that DSDCmd.exe is not setup to accept stdin instead of a file, so combined download/decrypt is not possible).

YES, IT WORKS!!!! End to end, I was able to download with TS to .TIVO and decrypt to .mpg which can be played in VLC with captions and the mpg uploaded back to the Tivo with the same beautiful captions as the original. THANK YOU SO MUCH FOR ADDING THIS AND FOR ALL OF THE ASSISTANCE!!!

This is a complete win for my goals. I am going to test comskip which is still running but I'll be quite content even if that doesn't work out. I'm about 4 days into the 15 day VideoRedo trial so it's available if needed.

Quicktime player does not show the captions, but that's okay. I have your "captions" task queued and will put the .srt file in the same folder as the .mpg just to test, but VLC is probably a better player anyway.

Testing so far has been successful on two recordings which failed every other approach, and their upload succeeded to two different Tivos.

After my other tests I'll copy my Parallels virtual machine and put it on a second Mac to go into production mode with the 2 Macs to download all of my recordings to date. Hopefully never again will I have to be concerned about filling the Tivo hard disk. And I can take the mpg's with me to play on the road.

It's so nice to have everything working in a single mpg so I don't have to save every .TIVO (though I probably will as a form of backup. :-) )

As always, your instructions were perfectly clear and complete. You have a great talent for not only imagining what needs to be done and writing the code but also for explaining it clearly. THANK YOU, THANK YOU, THANK YOU!!

I'll use this for a few days and clean up my notes and come back with any new results.

BUT MISSION ACCOMPLISHED for what I really wanted to do. Anything else is gravy.

BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.

William, thank you so much for joining the conversation. You and Kevin are doing wonderful things for humanity. Many thanks to you both!

YES, IT WORKS!!!! End to end, I was able to download with TS to .TIVO and decrypt to .mpg which can be played in VLC with captions and the mpg uploaded back to the Tivo with the same beautiful captions as the original. THANK YOU SO MUCH FOR ADDING THIS AND FOR ALL OF THE ASSISTANCE!!!
Regards,
-deac

First failure - a little more testing shows that the mpg's often only have about 70% of the captions of the original recording. The .TIVO files when uploaded so far do have perfect video, audio, and captions, so I will be keeping both .TIVO & mpg.

I suspect that the DirectShow Dump may have some. I understand that it's no longer being supported, that would be a shame because it seems powerful.

I just started a new test using VideoRedo to decrypt. I'll see how that comes out.

I just started a new test using VideoRedo to decrypt. I'll see how that comes out.

After a tiny sample it appears that VideoRedo only misses a few sentences in the captions and then picks up again. And the captions are much less noisy than with DirectShow Dump. Unless you can think of a way to get DirectShow Dump cleaned up, buying VideoRedo may be the best system.

deac33, it doesn't really make much sense. Both VideoRedo and DSD are using the EXACT same TiVoDirectShowFilter.dll from TiVo Desktop to decrypt the .TiVo file, so decrypting with either one should give same result. The difference if there is indeed one is that if you are using kmttg set to VideoRedo for decrypt, it runs decrypt + qsfix in one shot, in which case it's cleaning up glitches in timestamps compared to the .mpg produced by DSD. But if in kmttg flow you use DSD to decrypt and then run qsfix using VideoRedo you should get the same .mpg file as a result.

Good point. Yes, I have one video that fails (or not) in a given place making comparisons easy; overall I'm trying to keep my testing as consistent as possible, changing only one parameter at a time.

Quote:

Originally Posted by moyekj

deac33, it doesn't really make much sense. Both VideoRedo and DSD are using the EXACT same TiVoDirectShowFilter.dll from TiVo Desktop to decrypt the .TiVo file, so decrypting with either one should give same result. The difference if there is indeed one is that if you are using kmttg set to VideoRedo for decrypt, it runs decrypt + qsfix in one shot, in which case it's cleaning up glitches in timestamps compared to the .mpg produced by DSD. But if in kmttg flow you use DSD to decrypt and then run qsfix using VideoRedo you should get the same .mpg file as a result.

Great info, thanks.

My plan is to just test just the decrypt step starting with a known good .TIVO file using FILES tab with settings 1. and 2. here:

--------
However, I'm now unable to get a new .TIVO file, I've tried 5 times to do just the download which has succeeded before but now I only get the first 3 minutes of the video. The original on the Tivo box still shows good all the way through. I have no clue why this new glitch came up.

BUT, as soon as some other jobs finish I'll reboot the virtual machine, reboot the Mac and try again.

Note on your test #2: Even with just "decrypt" checked when you configure kmttg to do decrypt with VRD, it will always run qsfix regardless if you check "qsfix" or not. That's why I suggested if you want to compare both, then for DSD test #1 you should check both "decrypt" and "qsfix".

BTW, there is no question that VRD is a superior solution to just having DSD, because qsfix fixes a lot of issues that just DSD alone won't. So you should at least spend the money for the $50 version if not the $100 version.

Note on your test #2: Even with just "decrypt" checked when you configure kmttg to do decrypt with VRD, it will always run qsfix regardless if you check "qsfix" or not. That's why I suggested if you want to compare both, then for DSD test #1 you should check both "decrypt" and "qsfix".

Re-reading your earlier message I see that you did say that before. Thanks for repeating. :-)

Quote:

Originally Posted by moyekj

BTW, there is no question that VRD is a superior solution to just having DSD, because qsfix fixes a lot of issues that just DSD alone won't. So you should at least spend the money for the $50 version if not the $100 version.

Is qsfix only available with VRD, I see the box on the front panel for QS Fix is enabled on my computer without VRD, but is it a no-op without VRD?

So using DSD plus QS Fix is essentially (or exactly) equivalent to using VRD decrypt?

I see in the VRD tab in Configuration the choice for using VRD AdScan, is it usually a bit better than com skip?

I am inclined to take your suggestion and buy VRD. It's great to have it in trial version for now.

(My downloads are working again and my testing is proceeding. Things are generally looking pretty good.)

qsfix without VRD uses ProjectX which is free but nowhere as good as VRD and totally useless to you as it throws out captions completely.

Right, I had in the back of my head something I had read about qsfix not working for what I want. But the VRD version of their qsfix does what I want. Good, another reason to buy it.

Quote:

Originally Posted by moyekj

I don't use commercial detection but Comskip is much more advanced than VRD AdScan and can be customized for different shows.

DSD + VRD qsfix is not identical to just VRD qsfix for decrypt but should yield same results, but if you have VRD anyway then it's pointless to use DSD and make a 2 step process instead of 1 step.

So I'll settle in on using VRD decrypt. Didn't you say that with VRD chosen for decrypt that automatically uses its qsfix also, so would it be true that it doesn't matter whether the QS Fix box is checked?

So I'll settle in on using VRD decrypt. Didn't you say that with VRD chosen for decrypt that automatically uses its qsfix also, so would it be true that it doesn't matter whether the QS Fix box is checked?

Yes, with VRD configured for decrypt, if you check "decrypt" and "qsfix" only a "qsfix" job will be run (since qsfix will decrypt anyway). Similarly if you check "qsfix" and not "decrypt" it will also just run "qsfix".

With VRD configured, the "AdCut" task also uses VRD, else projectX would be used for that thereby throwing away your captions again. VRD is just a great tool to have period.

Kevin,
Thanks again for holding my hand through all of this. I'm in production mode for downloading and fitting in some testing. Results are totally good enough to accomplish my mission, thank you so much!!!!!

Captions are holding up extremely well, mucho thanks for that. My hearing is just bad enough that it really helps me to fill in dialog and that would otherwise be missed. Yayyy for you and captions!!!!

I have a couple of sporadic problems that I'm still testing, but if you have any insights I'd love to hear them.

1. Some mpgs upload with "Description not available" when I press the Info button on the Tivo remote. It appears that the .TIVO upload does still have the info however. Not a big deal since IMDB has all of the info.

At times the Program Details says "... container mpegts not compatible...", not sure what that means and if it's related to "Description not available". The video does upload and play fine, just no description.

2. Still sometimes get "Transferring prohibited by the copyright holder". I seem to recall that was usually a problem with old ffmpeg, but I have 2.2.1 which is the latest on ffmpeg.org and works with most recordings.

I just downloaded 2 new videos and in trying to upload them back both of them failed when attempting to upload .TIVO but the .mpg does upload (without Description, but it plays, and with captions). Strange, no?

All is well here thanks to you. I am very indebted for all of the time and expertise you put into helping me get what I was looking for, thank you very much.

I'd love to invite you and a guest to a fine dinner at a restaurant of your choice. Or send you a gift card for same, any way we could arrange that. Please drop me a message at deac333@gmail.com so I can arrange it.

You are an excellent and patient instructor. Thank you for all of your help.

I just wanted to pipe in here and add that I am presently dealing with the exact same corruption issues with captions on videos downloaded from Tivo units.

The corruption to the captioning is exactly as you have described and oddly, it has grown much worse in the last couple of months. I have downloaded videos from Tivo units for years and have never seen them this bad.

I am grateful that this thread seems to produce the answer as to the "why" - program .PS downloading seems to be the culprit.

I read through this thread with great interest and my only disappointment is that the solution here involves a workflow that uses a windows emulator.

I will keep this thread bookmarked but in the meantime I am keen to investigate further and see if there is *somehow* a solution to be had that can be 100% OS X (i.e. without resorting to windows emulation).

My goals are similar to deac33 but differs in that I am not looking to return videos back to Tivo units, but rather to convert/encode them to videos with hardcoded subtitles for viewing on the various iOS and OS X devices around the house (Apple TV, iPhone, iPad, Macs, etc)

By the way, great thread guys - if there was a /bestof/ category on tivocommunity.com I'd nominate this

... see if there is *somehow* a solution to be had that can be 100% OS X (i.e. without resorting to windows emulation).

My goals are similar to deac33 but differs in that I am not looking to return videos back to Tivo units, but rather to convert/encode them to videos with hardcoded subtitles for viewing on the various iOS and OS X devices around the house (Apple TV, iPhone, iPad, Macs, etc)

By the way, great thread guys - if there was a /bestof/ category on tivocommunity.com I'd nominate this

I'm glad to hear someone else got as much value from this thread as I did. I don't know how much time Kevin spent on his responses, but they are great. We should put them in a book since they cover so much.

You're right that I now use Parallels to get the Windows solution, but I did try Mac only approaches for probably two months first. I have a vague recollection that it was my need to re-upload the files back to the TIVO that caused me to resort to the emulator approach.

I'm not sure, but I think I may have succeeded with cTivo to do what you want to do. I'll try it again and see. It seemed to be pretty easy to get mp4 files that run on the iOS devices, I'll have to re-do some to see if they retain captions. I think the TS container is only required to re-upload, but PS should be okay for mp4's on the Mac itself and iOS devices. I think. We'll have to check.

I just dug out an old mp4 file I'm almost sure was from cTivo, it does have captions on the Mac and I just made sure that putting it i iTunes transfers to my iPod and plays with captions. Please give that a try. Please post your results, good or bad.

With any luck at all Kevin will also see your post and be able to add clarity on your needs using kmttg. I wouldn't be surprised if that can be done using OS X only. (Hopefully).

BTW, I'm not sure what you mean by "hard coded captions". I just played my (I think) cTivo mp4 on my iPod and needed to click the caption button (looks like an old comic book dialog cloud). But it showed the captions well.

I was actually successful in working out a workflow that allows me to create videos using cTivo solely in a OS X environment.

However, for the occasional video (which is occurring with increasing frequency) where the entire .SRT file is corrupted (the same corruption issue that you shared earlier in this thread), i have found no solution yet. For some reason this issue occurs with some channels but not others. I am not sure if this has anything to do with it, but the videos where I have no problems tend to be 30 minute shows, whereas, lately the movies I have downloaded have been badly corrupted.

I determined that the corruption was occurring on the source .tivo file, so it could not be a decoding/ccextractor/encoding issue. There is something going on that is causing this while the video is being downloaded to the computer.

I tried something just now and thought I'd share my results.

I selected one movie that has the severe captioning corruption problem when downloaded (both on KMTTG and cTivo), and tried downloading it, this time as a .ts Transport Stream file.

As expected, KMTTG could not decode it to .mpg, so I had just the raw .tivo file to work with.

I ran the raw .tivo file through ccextractor with the -ts (to read transport stream captions) option, and it created a .srt file, which I examined and found was ALSO corrupted badly. I do wish I could decode the .tivo file and see if it might generate a .mpg with a less corrupted caption/subtitle track, but i think this is where it becomes necessary to have a windows emulator.

I was hoping that a transport stream download could produce a perfect .srt file so that my theoretical workflow could be adjusted to :

1) download the video as PS (program stream), for further encoding, and then
2) download the video again, this time, in TS (transport stream), for the sole purpose of generating uncorrupted .srt file, after which I could toss the TS video download.
3) encode the video generated from #1, along with .srt file generated from #2

I know it sounds terribly inconvenient, but if that idea worked, I would have been happy as it is better than anything I have now for dealing with these videos with corrupted caption tracks.

I determined that the corruption was occurring on the source .tivo file, so it could not be a decoding/ccextractor/encoding issue. There is something going on that is causing this while the video is being downloaded to the computer.

I tried something just now and thought I'd share my results. I selected one movie that has the severe captioning corruption problem when downloaded (both on KMTTG and cTivo), and tried downloading it, this time as a .ts Transport Stream file.

I ran the raw .tivo file through ccextractor with the -ts (to read transport stream captions) option, and it created a .srt file, which I examined and found was ALSO corrupted badly. I do wish I could decode the .tivo file and see if it might generate a .mpg with a less corrupted caption/subtitle track, but i think this is where it becomes necessary to have a windows emulator.

I was hoping that a transport stream download could produce a perfect .srt file so that my theoretical workflow could be adjusted to :

1) download the video as PS (program stream), for further encoding, and then
2) download the video again, this time, in TS (transport stream), for the sole purpose of generating uncorrupted .srt file, after which I could toss the TS video download.
3) encode the video generated from #1, along with .srt file generated from #2

I know it sounds terribly inconvenient, but if that idea worked, I would have been happy as it is better than anything I have now for dealing with these videos with corrupted caption tracks.

Yes, now I recall that my caption corruption problems with cTivo were intermittent. I like your analysis that the corruption seems to happen more on certain channels and more on full length movies than the 30 minute shows. Also your attempts to download with TS to see if you could get a clean .TIVO file with captions was insightful. Good thinking, sorry it didn't work out.

It appears that you've narrowed it down to an inability to download a clean .TIVO file with captions using either cTivo or kmttg and either PS or TS.

I'm sure that you did check that the original recordings do play on the Tivo box with captions to show that the original is good, yes?

I think I'm at the limit of my knowledge, sorry. You've done more than I would have thought of. All I can suggest at this point is to see if Kevin picks up this thread again with ideas on how to better get a clean .TIVO file on MacOS kmttg. If not, then perhaps starting a new thread with many of the notes you've posted here might reinvigorate the discussion.

I'll keep watching and try to contribute if I actually can think of anything useful.
Best,
-deac

Perhaps I should create a new thread to track this corruption issue here, but I just wanted to respond to your questions and add a new discovery I just made that might be of interest to you:

1) Yes the Tivo recordings have captions that display properly without corruptions on TV.

2) I have narrowed it down somewhat, yes. The corruption is occurring either during the transfer from the Tivo box to the computer although I do not know if it is the Tivo box's OS software that is causing it or the Tivo decoding process (i.e. perhaps an issue with how tivodecode processes the caption track). I do know that the same issue occurs both with KMTTG and CTIVO.

3) I recently recorded a movie from the same channel that has consistently produced corruption issues (Cartoon Network), but this time I recorded the movie from Cartoon Network's HD channel. After I downloaded the HD movie, I see that it produced perfectly uncorrupted captions.

Up until now, I have always recorded animated programming from analog channels with no issue - as I mentioned the badly corrupted captions are a recent phenomenon. I will now record from HD channels going forward, and see if that "fixes" the problem. It would be interesting to see if the corruption is due to some change to how analog caption tracks are stored and/or transmitted.