Technology, music, and travel from a New York City IT consultant and musician.

Saturday, March 14, 2009

converting DVR-MS/MKV to iPod

As I mentioned in my prior post, I have been "broadcatching" HD content in x264/MKV format, and converting to DVR-MS to play on my extender, and so far I've been pretty happy with it. I do have to keep my eye on things more than I'd hoped but it's working fine.

However, I had to find a new way to convert downloaded MKV content to the iPod. (A 30GB "iPod Video" or "5G" device. It's a couple of years old but it has worked fine, and my birthday's coming up in five months and the iPod Touch will probably be reasonably priced or replaced with something even more technologically awesome, if my dear readers get the hint :D ) It so happened that I came into possession of a bunch of MKV files for a 10-year-old show I've been watching a lot of, with no XviD copies available online.

First attempt: SUPER

Thus I turned to SUPER as it is free and seems to convert anything to anything else one you figure out its many options. My first try used these settings, which I, for convenience's sake, exported as iPod_Video:

Output container: mp4

Output video codec: MPEG-4

Output audio codec: AAC LC

video size: 320:240

aspect: 4:3 (as I edit this after the fact, should've been 16:9)

frame/sec: 30

bitrate: 1248 (in retrospect, could've been lower)

options: Hi quality, top quality, stretch it

(audio) sampling freq: 44100

channels: 2

bitrate: 96

(I know it would've been easier to show the picture, but I don't want to confuse my family by using my flickr account which is typically used for showing my entire extended family and friends pictures of my son.)

These settings resulted in a 54MB file that had no sound (in WMP, I didn't check iTunes/Quicktime). Several other attempts (that I cannot now remember, as it's been five days since I started working on this) were similarly slow and unsuccessful.

Second try: iPodifier/ffmpeg

I then downloaded iPodifier and set it up to watch the Recorded TV directory, figuring that MKV files were automatically converted to DVR-MS via the method documented in my previous blog post, and then iPodifier could pick them up for conversion to iPod Video-compatible MP4.

Seemed simple but, when I launched it, this showed up in the log window over and over:

According to the wiki this is related to some sort of problem that mencoder is having with the DVR-MS file causing an infinite transcoding loop, the first of its known issues. I am having the second known issue but had already renamed the files, so I just tried with a renamed file. Same thing:

I had to remove iPodifier's suggested "threads auto" option as that doesn't seem to work on my relatively recent build of ffmpeg (SVN-r16596-Sherpya). iPodifier was also making a mess of the filename so this was about the point at which I gave up on iPodifier and just used command-line ffmpeg.

The size was still tiny (70MB) and you could tell in iTunes that the quality was much worse than my ~320MB episodes converted from XviD using my old method.

I tried again with qscale 4 (iPodifier's "Best" quality) and the result was a 107MB file in ~44 minutes. I didn't even watch it (because I overwrote it accidentally in the next run).

I tried again with qscale 3, having found this article and noting that it seemed to be a better tradeoff between quality and size. Its result was a 140MB file. On this one, I noticed that in both QuickTime and when synced to the iPod the audio goes out of sync pretty early on, and also that the rendered video size was actually 426x240, not 320x240 as I had asked it to convert to. I guess I don't know what it ought to do in this case, but the program I've been using gives me a letterboxed 320x240. iTunes accepted the import of the video and the iPod showed it properly, but maybe the other program (which typically yields a ~330MB file) is inserting the black bars into the material itself, hence its larger size? Seems silly given that iTunes (or maybe the iPod itself) can handle the conversion.

Pressing on, I tried again with qscale 2, resulting in a 214MB file, and tried again with qscale 1, which created a 721MB file. Both looked great, not noticeably different from each other, but it was with this run that I noticed that both files had out-of-sync audio (though qscale 2 was further off).

So I decided to stick with qscale 2, but needed to dig into the settings to fix the audio. Here was the command line I started with, courtesy of iPodifier (minus the "threads auto" option that didn't work in my relatively recent build of ffmpeg):

My guess was that the "async" parameter was the problem. I also imagined that dts_delta_threshold had something to do with it but I don't actually understand at all what the description ("Timestamp discontinuity delta threshold") means. Somebody fixed something at the bottom of this page by setting it to 0 so that may be worth a try as well. Yeah, I'm not that scientific with this shit. I tried async 1 and dts_delta_threshold 0, but the sound was immediately, horribly off.

This post suggested using async 2 so I tried that (with dtb_delta_threshold turned off). No luck.

I tried once more with async 2 and dtb_delta_threshold set to 1, but that didn't work for me, either. I decided that maybe MKV --> DVR-MS --> MP4 was not that efficient. But what converted MKV to MP4?

Third try: XviD4PSP

Good old ViDEOHelp.com. They offered like 8 ways to convert MKV to iPod MP4 and the simplest used XviD4PSP. Though I had no need for any of the formats described in the name of this free product, and was worried that most people are converting for use with newer iPods than mine, I figured it was worth an hour of my time to try given the many frustrating hours I'd spent on the other programs. On my first try, I used these settings:

Format: MP4 iPod 5.0G

Denoise/sharpen: disabled

Brightness/contrast: disabled

Video encoding: x264 HQ Ultra (yeah, it bothered me that only x264 and XviD options were present, but I gave it a try)

Audio encoding: AAC-LC ABR 128k

This worked! The result was 320x184 (which I guess makes sense given the 843 x 480 source resolution) and it was only a ~115MB file, but the play was very smooth and the sound was pinpoint-accurate.

I looked at the output in the XviD4PSP window to see if I could reverse engineer the command lines and utilities used, but I'm not sure. Clearly it is using the x264 utility for video encoding, and it does appear to output its settings, but I can't tell what it uses for audio or for muxing video and audio back together.

The major downside of this utility is that it doesn't appear to have a queue -- it's just one video at a time. This is exacerbated by the fact that it is much slower than DVR-MS to MP4 via ffmpeg -- that took 7 or so minutes for the file I was using to test, whereas XviD4PSP takes about 30 minutes for that same file. So, I can't queue up a bunch before I go to sleep and have them waiting for me in the morning. Nor can I schedule them until I figure out the underlying utilities that provide the actual conversion.

Update: I received a comment pointing out that there IS a queue -- you load each file and then click Save, and then you can load another, and so on until you click Encode to convert the whole batch. Frankly I'm using EncodeHD now which supports drag and dropped files, and it does all its work through ffmpeg so it's pretty fast.

But, it works and the quality is good. After a week of trying other methods and failing, that's saying a lot.

OK I know this is an ancient article, but I just came in thru Google looking to see if anybody's smart enough to know that 1. XviD4PSP > anything out there, and 2. to port XviD4PSP to the Mac using Codeweavers' Crossover. Your blog came up on the 1st page. So.

Anyway regarding XviD4PSP, the video tool I swear by, you wrote here:

"The major downside of this utility is that it doesn't appear to have a queue -- it's just one video at a time."

There is no major downside. There is no downside to speak of. (That is, I'm assuming you're using the proper version, which is 5.036.) The queue you speak of is performed by clicking on SAVE instead of ENCODE button. You click save on every encode job, then when you're satisfied, you click encode. It's that simple.

thanks for your comment. I'm pretty sure I did figure out that "Save" meant "Enqueue" and I used Xvid4PSP for awhile.

but at some point I found EncodeHD and started using that as a quick way to convert stuff to iPod and also XBOX, and through its log, I discovered all the command line options for ffmpeg to do a reasonable conversion. so now I have automated ffmpeg scripts that convert everything and I haven't used XviD2PSP in at least six months.

I'll correct the blog entry but I feel a bit ambivalent given that I'm doing things so differently now. but you keep on loving your XviD2PSP for me.