Compressor Frame Rate Conversion is very, very slow

I need to convert a whack of 1920X1080 XDCAM EX files from 29.97fps to 23.98. (I know it's not ideal, but the shooter messed up and I'm stuck cleaning up the mess as everything else is in 24P).

It's about 140 files, with about 2.5 hours total, as most are quite short b-roll shots. They were provided on a pretty dinky FW800 drive -- a 2 TB "WD My Passport Studio." It's not a fast editing drive, though it has been working fairly well for basic editing needs within FCP 7 so far.

I considered using MPEG Streamclip, but as this footage is for a broadcast doc, I wanted to get the best quality I could. So I am using Compressor with "better" rate conversion and de-interlacing settings.

Big mistake. It's been now running for about 40 hours, and only half the files are transcoded. The status bar says 30 hours left. But it said 25 hours five hours ago and it just keeps getting higher. At this point I see it being done some time on Tuesday or Wednesday. I'm running out of the allotted time for this project, and am not sure what to do.

Questions:

1) Is it likely the slow external drive that's creating this backlog? Would a 7200 RPM drive make a difference? Or one with a faster connection like TB or USB 3.0 work better (moot because I don't have ports for either, of course)?

2) Is it more likely just my older MacBook -- a 2.5 Ghz Intel Dual Core with 4GB of RAM? It's been a workhorse, but it's starting to show its age.

3) Would cutting my losses and switching to Streamclip make a visible difference in the quality of the transcoded product? I tried doing a test transcode of a couple files in Streamclip and it looks OK to me...

[Ben Singer]"So I am using Compressor with "better" rate conversion and de-interlacing settings."
Wrong.
Set BEST for deinterlacing, but set "Next Frame" (linear) for the time-base conversion.
When going 29.97 to 23,98 the only thing needed is to drop some frames.
if you set other than "Next Frame", all the picture will be reprocessed.
The process will takes long and the image will lose a generation.
rafael

Thanks Rafael. That's interesting. My understanding was that it's not just "dropping frames" per se, but blending them to turn 30 into 24 without making the motion super weird. I had also never heard about "losing a generation" in the context of using Compressor with digital files. I thought we escaped all that when we abandoned tape.

[Ben Singer]"My understanding was that it's not just "dropping frames" per se, but blending them to turn 30 into 24 without making the motion super weird."
Well, you have to chose.
Dropping frames may make the motion more choppy, but will keep the picture pristine as the original.
Every single frame is as on the original movie.
Blending frames may makes the motion smoother, but destroy the picture.
Only one in every 4 frames remains untouched. The rest are blended.

I propose an easy and fast test:
- Take a short (2 or 3 seconds will be enough) 29,97fps clip in FC and apply the "TC Generator" or "TC Reader".
- Export from FC and bring it to Compressor.
- Convert to 23,98fps using the different options; Fast, Good,
- Open the clips in QT Player and go through the picture frame by frame.
Have a look to the last to digits of the TC.

[Ben Singer]"I had also never heard about "losing a generation" in the context of using Compressor with digital files. I thought we escaped all that when we abandoned tape. "
Whenever you process a QT movie, there is a generation lost; unless you export to n Uncompressed format.
rafael

Thanks for your help earlier Rafael. I think the footage looks good compressed using your idea.

Now I have a slightly different question. Most of the footage was in 23.98, with about 1/3 or less in 29.97, or probably 1/4 of what will be in the final film.

I chose to convert the 29.97 stuff to 23.98 to get everything into one frame rate, and am working with that in FCP 7. Now I am told I need to deliver the final product in 60i (29.97 fps) to the broadcaster. :S

Now my question is: do I keep editing in my 23.98 timeline with the footage in that frame rate, and worry about converting to 29.97 at the end? My concern here is mostly with the 29.97 stuff I already compressed to 23.98. When that gets REconverted back to 29.97 at the end, I worry how it will look.

OR, do I go back and convert all the 75% of the material that is 23.98 over to 29.97, and cut that way. At least this way there won't be much lost in the FINAL conversion, because the cut will already be in 29.97. But this could cost me another 3 days in converting files, which makes me want to cry.

OR, do I forget about converting anything for now, cut with multiple frame rates, using whatever the material came in as originally, and worry about the conversion at the end. I hesitate because from past experience I get constant rendering issues in FCP 7 and I fear my edit will drag and drag.

The point is that by using the FAST (Next Frame) option in Compressor, the picture is not re-copressed, and the interpolated frames are build not by repeating frames, but by putting together one field from each adjacent frame. That makes the picture look smoother.

You can deliver your 29,97fps master just like that, or if you have time and feel like doing a second edit, replace in the master the footage what originally was 29.97fps.
Not sure if is worth this extra step.
rafael

[Rafael Amador]"When going 29.97 to 23,98 the only thing needed is to drop some frames."

I was going to jump in with a big "whoa there!", but the discussion has progressed.

Correctly, you both have come to the conclusion that you need to make a choice. In this case, going from original 29.97 to 23.98 is not at all straightforward. No, the math does not work out, and the choice will be between spatial and temporal resolution.

You will either end up with a quickly achieved, but compromised, output, or a slow, but high quality result -- if you choose a software process. Find someone with a hardware processor if you need "Fast" and "Good", but "Cheap" may not be available, as per usual.

I seem to be doing frame conversions on a regular basis now. (Have Teranex, will convert.) So, in this case, having built it, they are coming.

Alos taken aback somewhat by the relative ignorance about frame rate that so many digital camera operators are exhibiting, but its mostly confusion about what the camera is producing and what the recorder is actually putting in the file.

jPo

"I always pass on free advice -- its never of any use to me" Oscar Wilde.