Article focus:
Looking at traditional methods of time-remapping, frame-rate conversions and other "spatial continuum" methods, we'll see how the popular Twixtor plugin compares to the task at hand. The results are interesting with some surprising results. We'll also look at the ways Twixtor works well with its sister plugin, "ReelSmart FieldsKit". Those looking into film-look conversion techniques will especially benefit from this review.

Developer Spotlight:
It's safe to say that Pete Litwinowicz (pictured) and Pierre Jasmin, the creators of the ReelSmart plugins are a couple of pretty smart guys. They've developed a process in which their plugins look at pixel information and interpolates new data based on morphing algorithms as opposed to traditional frame-blending methods. And best of all, the new data that is created is formulated from areas of the image that need it, leaving the areas that don't need it untouched, thus preserving the integrity of the shot with as less interpolation "loss" as possible. Now that's "Reel Smart"!

About Twixtor

Let's first look at the basics surrounding the Twixtor plugin. You'll be hard pressed to find an application that won't run Twixtor. Supported applications include Apple Final Cut Pro, Apple Shake, Adobe After Effects, Adobe Premiere, Discreet Combustion, Discreet Flame, Avid, Pinnacle Commotion, Quantel generationQ and Eyeon Digital Fusion. Supported operating systems include Mac OS 7.5 to OSX, Windows 95 to XP and Irix 6.5 and up. Now that's compatibility! For the full low-down on product support, visit the RE:vision compatibility page. The process of this review will be using Twixtor in Adobe After Effects 5.5 on OSX.

Twixtor comes in two flavors: a standard version ($329.00 USD) and a Pro version ($495.00 USD). Note that some applications (e.g., Shake and Avid) only offer a Pro version with slight price reconfiguration. Some applications also support Render Node support ($165.00 to 198.00 USD). There are also upgrade offers from the Standard version to the Pro version ($165.00) for applications that support the Standard version.

The Pro version adds 16-bit rendering support (for applications that support it, like Adobe After Effects), which to me is the biggest reason to go for the Pro version since I work with nothing but 10-bit uncompressed files in 16-bit After Effects projects in "Trillions mode". Also in the Pro version is the ability for Twixtor to assign the foreground/background setting with a matte so Twixtor knows which portion of the image comprise a foreground object(s) and which areas comprise a background object(s) that move in dissimilar directions (including the possibility of one object(s) staying static). This function can prove vital for those really tough clips with intricate motion circumstances.

The "prime cut" to this beefy plugin is it's ability to change the speed of a clip (either constant or keyframed) and to change frame-rate speeds. As noted from the RE:Vision Effects website:

ReelSmart Twixtor enables you to speed up, slow down or frame rate convert your image sequences with visually stunning results. In order to achieve its unparalleled image quality, Twixtor synthesizes unique new frames by warping and interpolating frames of the original sequence... employing RE:Vision's proprietary tracking technology that calculates motion for each individual pixel.

This is very important to understand because Twixtor doesn't merely frame-blend the motion together like traditional methods (although the option to do that is there).

Getting started with the plugin

The first thing you'll encounter when using Twixtor for the first time is its inherently high learning curve. Even seasoned compositing pros might find the plugin a little daunting at first. But rest assured that given the time and patience, you'll soon grasp the concepts, functions and power of Twixtor. It's really a matter of getting familiar with the plugin terms and function relationships/differences. Download all the Twixtor tutorials, study and fiddle around with them. I found this the best way to get accustomed to the plugin as my initial ideas of controlling the parameters were way off target. Don't waste time trying to figure it out on your own like I first did... download the tutorials, read the manual and you'll be much happier in the end!

The biggest thing to understand when using Twixtor is how to initially treat the footage you import in. For example, when you import a clip into After Effects, the default settings for the clip will most likely be set to either "Upper Field First" or "Lower Fields First". You must change the Separate Fields setting to "Off" for Twixtor so it can fully use both fields of information from the clip. My initial lack of desire to read the manual and play with the tutorials (but over ambitious desire and excitement to just start "playing with the plugin") transcended into results that I referred to as, "ewww... why does this look so bad?!!" In fact, failure in going through the manuals and tutorials may make you utter words like, "man, this plugin sucks!" In reality, proper education will result in words like, "man, this plugin rocks my world!"

Next on the "to-do" list of items is deinterlacing the footage. Even if you're bringing interlaced footage in and spit it back out to the same exact frame-rate with interlacing, Twixtor still deinterlaces the footage in order for it to properly work its magic. This can be done in two ways. The easy way is to have Twixtor deinterlace the footage using its own simple deinterlacing function. The best way (and the only way I do it) is to use RE:Vision Effects ReelSmart FieldsKit. This will add to the already long render time you'll encounter, but it will be worth your efforts as the quality is bar none.

As I mentioned earlier, deinterlacing your footage is required in order to work with the Twixtor plugin. This process occurs even if your input and output settings are set to Lower or Upper Field First in Twixtor's settings. If you're using Twixtor to simply alter the clip's speed, then you should match your current clip's attributes to the other footage based in your overall project. With NTSC footage, this means setting the Input and Output Frame Rate to 29.97 with Upper or Lower Field set appropriately on both. Changing the "Output: Fields" setting to None will output your effect with no interlacing (as shown in the image). Some people like this non-interlaced flicker effect as it reminds them of a film-look process (more on this later in the article).

For the most accurate deinterlacing methods, let's look at RE:Vision Effects' ReelSmart FieldsKit plugin set. FieldsKit is composed of three distinct plugins; Deinterlacer, Pulldown and Reinterlacer. We won't be using the Pulldown or Reinterlacer plugins in this review, but we will be using the Deinterlacer plugin in association with Twixtor from this point on in the article. The two plugins together are a perfect marriage for optimum quality and I personally wouldn't do it any other way. So let's get a closer look at Deinterlacer.

The screenshot of the FieldsKit Deinterlacer plugin shows the default settings. Here's our goal: we want to deinterlace the footage in a way that will maximize the available amount of information that is currently in our source clip's 29.97 interlaced footage without throwing out the little information we have while also not introducing unneeded interpolation. The best way to perform this is to take each even and odd field group and turn them into their own frames. In other words, we want the upper field to create a full new frame and the lower field to create a full new frame and so on. We can do this in two ways... doubling the clip's duration, or doubling the clip's frame rate. Opting for the latter method simplifies things so we don't have deal with duration conversions. So let's get to work.

Import your footage. Like the earlier Twixtor example, control-click or right click the clip in the AE Project Window, select "Interpret Footage", click "Main..." and change the Separate Fields setting to "Off". Create a new comp and make the Frame Rate of that comp 59.94 FPS. Place your imported clip in the AE Timeline. Apply the Deinterlacer plugin to the clip. Let the dog out to go pee. Oops! Thinking out loud here!

Now we can start tweaking with the Deinterlacer settings. Change the Timing Mode setting to "2X FPS, 1 Frame Per Field". Since we doubled the comp's frame rate to 59.94, we'll need to use the 2X FPS option. Select the proper Field Order in accordance with the type of clip you are using. For me, I'm using an AJA Kona SD for this review so my setting is Lower First. My Aurora Igniter system on the other hand uses the Upper First setting (note to self: call up all the manufacturers and beg for a common standard).

So now we've created a 60p comp (59.94 progressive). This is incredibly valuable when dealing with any kind of time remapping because we essentially have an overcranked "clip" (precomp) to work with. So for example we want to create a 400% duration increase, we'll have a smoother result based on the extra frames of the 60p sequence. If we worked from a 30p sequence, we'd be creating frames based on further interpolation values, which in turn will result in a more "interpolated look." Think of this 60p technique as a higher "bit-rate" for time expansion! There are times when going from 60i to 60p isn't totally necessary and going from 60i to 30p is good enough. You'll get a better grasp of this with experience.

The next ten settings, from Fill Method to Sharpen are what you the user tweak to work the magic of the Deinterlacer plugin. Each type of clip will require different settings altogether. There is no "general settings guide" because each setting does something different and has advantages/disadvantages against the other settings. A clip of a head shot will require different settings than a clip with two trucks crossing their paths with text on the sides of them. This is a whole other article on its own! But for the sake of demystification, I sometimes have the following settings:

Fill Method : Blend or Best 5 Neighbors (sometimes 3)

Detect Motion: 3 Frames Compare

Detect Method: Large Area

Motion Tolerance: .35 (I've never gone lower than .30 or higher than .40)

Mask Suppress: 0 or 1, typically

Mask Grow: 0 to 5

Mask Feather: 0 to 5

Smooth Vertically: 0 but if it's called for, between .5 and 1

Sharpen: 0

Remember, these are general settings that get tweaked around on a clip-by-clip basis. Simply plugging these values into Deinterlacer on all your clips wont allow Deinterlacer to work at its full potential and may even make things worse. And USE the "View Motion Mask" option to see what areas of the image Deinterlacer is affecting. This is a very cool and powerful tool to use and lessens the guesswork in all of this. Play around with the settings until you get it right!

So how does Deinterlacer compare to other deinterlacing methods? Glad you asked. Let's just say the other methods don't compare in the least! Most deinterlacing plugins simply blend the to fields together resulting in loss in vertical resolution (although Deinterlacer does have this option should you opt for it). Below is a dynamic image comparison example of different deinterlacing methods. This is an impractical image in real world applications, but what the test elements do is challenge the deinterlacing plugins to some impossible tasks. The most impossible task is preserving 1-pixel lines. Why? Because the deinterlacing method takes 1-pixel lines (thinking they're interlaced fields) and blends them together. Because of this phenomenon, detail in sharp edges and text can also get tossed out the window through traditional methods. Another problem with traditional deinterlacing tricks is that a single object can double in appearance due to frame blending. An example of this would be a moving, talking head and instead of two eyes looking at you, you get four eyes trying to blend together... scary!

You'll need JavaScript active to make the comparison function operate.

Image Notes:

Source Image - This is the original clip running at an interlaced 29.97 FPS (lower fields first).

Single Field - This process simply uses one field only. This is the result you would get when capturing footage with only one field (as some have done for trying to achieve a film strobe effect).

Single Field Deinterlace - Same as above but adding the standard AE Deinterlace plugin to it.

AE Deinterlace Plugin - This is the standard AE deinterlacing plugin set to "1". For the pros: this plugin produces the same exact results as using two layers of the same clip, separating the opposing field-order on each layer and blending the above layer to 50%. Performing a difference test reveals this.

Twixtor Deinterlace - This is the result when using Twixtor's standard deinterlacing feature. Note that Motion Blur Amount was set to zero to aid in the deinterlacing of this wicked image.

FieldsKit Deinterlacer - Now we can start seeing some resolution back in the image. Although it's not perfect (as compared to a native 30p clip), it's still showing us it can maintain vertical resolution in tough situations. The problem with this test image is that I've purposely implemented different tasks for the plugin to attack. With FieldsKit, you would typically look at the overall image and asses the best type of setting for that task. Ah, but there's even a solution for multiple tasks...

FieldsKit Multilayered - For the true compositing nerd such as myself, you could easily use FieldsKit on multiple layers of the same clip (and masking only the areas of concern). As seen here, the detail in the right-hand grid images are in full swing. The model's hair and face are more natural looking and ALL the text and logo retain full detail without blurry after tastes. Another example of this would be a waterfall in the background with a truck passing from right to left of the frame with a logo and text on the side of the truck. The organic nature of the water in the background will require different settings than those needed for the hard lines of the truck and its text and logo. Of course a happy medium can be assigned, but the ability is there to multi-layer for optimal results.

Barend Onneweer, one of our esteemed Creative Cow leaders has a deinterlacing article here at the Cow that covers the different methods of deinterlacing video with After Effects. He too concludes that FieldsKit Deinterlacer is the best way to go.

Okay, enough on deinterlacing. Let's start really playing with Twixtor!

Frame rate conversions: 60i to 24p

One of Twixtor's powerful features is its ability to convert frame rates. The most popular method for this of course is converting standard 60i video footage to 24p. If you're new to the video world, the reason for doing this is to convert NTSC 29.97 FPS interlaced video to a 23.976 FPS progressive format, known as 24p. This 24p conversion process is part of the technique to make your video look like it was telecined from a film source since film is commonly shot at 24 FPS.

The basic process of converting 60i to 24p is three-fold.

Use FieldsKit Deinterlacer to convert 60i (29.97 FPS interlaced) to 60p or 30p. Precomp or sub-sequence the clip.

Bring the precomp into a new comp and apply Twixtor to the precomp and change the settings to input 60p or 30p and output 23.976.

Render your comp with 3:2 pulldown back to 60i (29.97 FPS interlaced). Those with 24p NLE support need not perform this step.

So let's dissect this three-step process.

Deinterlacing
Use the techniques mentioned earlier in the article to deinterlace your footage. I personally prefer to always convert to 60p as opposed to 30p since this will be beneficial to other areas of concern. You now MUST precomp this layer. If not, Twixtor will merely look at the image information, minus the deinterlacing process. So precomp this layer and place it in a new comp. Apply Twixtor to the precomp layer. Add 2 cups of sugar and follow as directed.

60p/30p to 24p conversion
This must be done correctly in order for you to achieve the best results. Remember how I mentioned the Deinterlacer plugin has different settings to achieve optimal results in different situations? Well Twixtor is the same way. You have different ways to perform interpolation, motion vectors, motion blur and motion sensitivity. You really need to tweak the settings to get the best results... no "general settings" here either.

The one thing that will always be the same is the way in which you convert the frame rate. Looking at the image, we can see I told Twixtor the incoming source is progressive (marked as "None") with a frame-rate of 59.94 FPS. I then told Twixtor that I need to keep it progressive on output but at a new frame-rate of 23.976 (as displayed by 23.98).

The thing that's tricky about Twixtor is avoiding a goopy mess when interpolating frames. Through serious tweaking of the settings, you can avoid this. Additionally, using the Pro version can really help avoid this by using a matte to specifically tell Twixtor which portions of the image need interpolating and which don't.

Below is another dynamic image comparison example of the finished 60i to 60p/30p to 24p conversion process by means of different methods.

The number represents the amount of images (not frames) to be seen in one second. E.g., in 60i video, there are 2 images for every frame of video (vertical resolution cut in half for each image... one image for even fields and another image for odd fields), thus a total of 59.94 "half" images are viewed every second. The letter after the number says whether those images are interlaced (half resolution) or progressive (full resolution). Note that the Twixtor manual says "30i" to represent 29.97 interlaced frames and does this so new users don't confuse 60i with 60 FPS. In reality, 30i means 15 FPS, interlaced.

*Note that this article uses 23.976 FPS for all 24p examples in accordance with NTSC compatibility standards.

Image Notes:

Source Image - This is the original clip running at an interlaced 29.97 FPS (lower fields first).

AE Deinterlace & Frame Blend - After Effects blends the adjacent frames together. In most cases, this produces a doubling of the image with the addition of added softness and excessive motion blur. Although this might appear to look more aesthetically pleasing as a still image, the result is most unfavorable on an NTSC monitor upon playback. In other words... YUCK!

Twixtor (no FieldsKit) - Motion really starts to shine, but the deinterlacing could use a little help. Cue, example 4.

FieldsKit & Twixtor - Once again, all the text and 1-pixel lines are kept in tact. There's no excessive motion blur or added softness. Most importantly, vertical resolution is maintained throughout. Viewed on an NTSC monitor, this result is by far the most favorable.

Going back to 60i and adding 3:2 Pulldown
There are several instances when going back to 60i with 3:2 pulldown isn't needed. They include finishing off to progressive DVD, CGI production, PAL conforming to 25 FPS or printing to film. Otherwise you'll need to go back to 60i video and introduce 3:2 pulldown. And if you don't perform the 3:2 pulldown, you're motion will have an ugly fast-slow judder to it. Some systems (like my AJA Kona SD and Aurora Igniter Film) can fully edit in native 24p, so rendering 3:2 pulldown isn't necessary (this puts a VERY large smile on my face!). These 24p compatible systems add the 3:2 pulldown on the fly on output to your monitor and deck. Totally trick.

The image below shows part of the After Effects Render Settings window to add 3:2 pulldown to 60i from a 24p timeline.

The process in whole looks something like the image below. Keep in mind this is a very basic diagram of the process and that other methods can be introduced (like conforming instead of interpolating).

What a freakin' mess! Obviously if you have means to a 24p camera... use it!!! Note also that the 30p conversion step can also be a 60p conversion process (my preference ni most cases) but is shown as 30p for simplicity's sake.

A little off topic: Conforming
Obviously the best way to get a 24p look is to shoot with a 24p camera. But if that's not possible, then shoot with a PAL camera running at 25 FPS. Then convert that 50i footage to 25p with FieldsKit Deinterlacer and conform that 25p conversion to 24p. By conforming, you're keeping all the frames in tact and only decreasing the speed by 96%... hardly noticeable by any account.

You can also conform your 60i to 30p conversion to 24p, which will reduce the speed by 80%, which is noticeable. This is great when audio is not part of the footage and you want a slight overcranking effect. The process slightly slows down the footage enough to give it a smoother appearance in motion.

Time remapping

It would only make sense that Twixtor would also be able to perform speed-change functions as well. Whether your speed is constant or keyframed for a "Matrix" type look, the results can be awesome. There are three "Stretching Methods" as Twixtor calls it; Constant Stretch, Keyframed Stretch and Speed. The "Stretch/Speed" value setting acts as they should. A setting of 2.000 in Constant Stretch mode will double the duration of the clip (don't forget about the solid layer rule in the beginning of this article). To keyframe a time remapping sequence, I really like using the Keyframed Stretch option. Make sure you also check your Motion Blur Amount setting... to little and it might look a bit jittery and too much may induce too much motion blur and possible "goop". This function also works in accordance with the Motion Sensitivity value. After you get used to these three different styles of time manipulation, you'll soon see the advantages it has over AE's time remapping system. And keep in mind too that you will not need to use any of AE's frame blending functions to make all this work. Twixtor and FieldsKit work their magic independently under their own superior algorithm systems.

In some strange instances however, I found that using AE's time remapping function with frame blending to be better suited... but this is on a rare occasion. I can't really explain it other than you need to try both methods out first before settling on a final result. More than likely however, you'll stick with Twixtor. I felt I needed to add this info because in my experience, Twixtor isn't the answer 100% of the time. And if time is a serious issue and you can sacrifice some quality (when Twixtor does show superior results) then using AE's time remapper with frame blending might be a better solution because Twixtor (especially in conjunction with FieldsKit Deinterlacer) is a serious render hog. More on that later in the article.

Despite the serious render times and monotonous tweaking one may have to succumb themselves to, the results can be truly amazing to say the least. I've taken the duration of clips up to 800% (that's a reduction in speed from 100% to 12.5%!) with KILLER results. They looked like they were overcranked at 192 FPS! The more organic and natural the shot is, the more realistic the in-between frames will be. Additionally, if you're strictly staying in an interlaced environment (60i for example), Twixtor performs an incredibly beautiful job of outputting extended durations with proper interlaced fields... something After Effects can only dream of. Even my above test image with an 800% increase in duration maintains detail in the blue jeans. Likewise, the bubbles on the left margin look natural and absolutely beautiful in every organic sense. Very cool stuff.

From spatial effects to special effects

You can get some really cool effects with Twixtor too. Using the Motion Blur Amount and Motion Sensitivity values can produce some neat results. Careful though... setting the Motion Blur Amount setting to high values can really tax your CPU. Below are examples of using these motion tools in a creative way.

Image Notes:

Motion Blur Amount: 10 Motion Sensitivity: 0

Motion Blur Amount: 10 Motion Sensitivity: 100

Motion Blur Amount: 3 Motion Sensitivity: 100

Source Image

The dark side of the mooooo

Pink Floyd aside, there is a dark, ugly and evil side to this beast. Fact is, Twixtor is a tough cookie to crack. I think part of it is due to the user interface. I was confused at first how the speed options worked. Maybe adding a percentage system would help. But in the long run, it's just a matter of getting used to the UI and with time, you'll get accustomed to it. I would really recommend downloading the demo and getting a feel for it. If it's just too much for you to grasp, then maybe sticking with the methods you already know may be best. And for goodness sake, download all the tutorials and manuals! There's even further help as found in the RE:Vision Effects CreativeCow.net forum with help from Pete Litwinowicz himself (the creator of the plugin). As complicated Twixtor might be at first, there's no excuse for not finding help with the forum and online tutorials. Seek and ye shall find.

The Jekyll to this Hyde of a plugin is it's a pig of a render hog. Render times will increase when you increase the Motion Blur Amount value, the Motion Vector Quality setting and a few other settings. The price of beauty takes its toll! Below are some render time comparisons using my PowerMac G4 DP 800. Note that all the render times for Twixtor include the processing time it takes to deinterlace the footage using FieldsKit Deinterlacer, so it's two-fold.

All processes use high quality switches and field rendering (for 60i output sequences)
All tests have a final duration of a 1-second TRT. Time results are in minutes.

Process

After Effects

Twixtor

Notes

200% Duration

00:16

03:51

60i to 60p to 200% to 60i

400% Duration

00:27

02:45

60i to 60p to 400% to 60i

Time Remap

00:16

04:10

1% to 100% to 1%

Frame rate Conversion

00:13

02:35

60i to 30p to 24p

Render farm anyone?

Closing Arguments

Yeah Twixtor and even FieldsKit Deinterlacer might require a degree in brain surgery to learn the nuances of the plugin, but the time to fully understand them will reward you ten-fold. I would say I use Twixtor about 95% of the time over AE's built-in time remapping functions... even with the high render time overhead. It's just so beautiful! And quite honestly, using Twixtor without FieldsKit is brainless. Buy both, you'll be happier. But like I said earlier, first download the demo versions to make sure you can grasp the concepts of these powerful plugins. Maybe in time we'll see a user interface update and render time improvements. Until then, it's still the best damn way to play time machine.

I can't give Twixtor a perfect 5 rating because the learning curve is high, it may require a lot of tweaking and the render times suck. Besides that however, the accuracy and fluidity of the final piece can have the potential of sheer amazement. I highly recommend this plugin to those seeking absolute superiority in time frame manipulation.

Pros: Very accurate. No image doubling. Vector based morphing algorithms. Could pass for overcranked film.

Cons: Slowwww rendering times. Intricacy can be daunting at first. Usually requires a lot of tweaking.

Marco Solorio is a multi-award winning digital media producer in the San Francisco bay area. He owns and operates OneRiver Media, which focuses on producing animated content for broadcast, as well as serving production needs for content developers. And of course, SpongeBob SquarePants is his third most favorite cartoon in the world.

The review was really very interesting and informative; just the sort of thing I need to help me with this topic of deinterlacing and not reducing video quality. I was sorry that I could not get any of the graphics to work even though Java script was enabled on all of the computers I tried. I think you have an error in your java script. is it possible for someone to give this a look over?
Thanks
John

Storage and data management company NetApp announced a set of tools to enhance cloud services, including a more efficient operating system with simple tools to manage data on Amazon Cloud web services.

Kyle Andal and Gerard Andal, lead compositors at Zero VFX, bring invisible post-production magic to several films - including 2014 Academy Award-winning "American Hustle," in which the team helped transform 2013 Boston to 1978 New York - and also several commercial shots with mocha Pro.

LightGrid Script and Demo Tutorial Part TWO with New FeaturePlay VideoLightGrid Script creator Brian Charles demonstrates his newly modified After Effects script for rigging lights in grids and circles. The modified script with the new feature, which adds the ability to enable individual light colors through a checkbox in the effect control panel, is available as a free download in the Project Files folder.

LightGrid Script and Demo TutorialPlay VideoLightGrid Script creator Brian Charles demonstrates his After Effects script for rigging lights in grids and circles. The script is available as a free download in the Project Files folder.

With multiple VFX Oscars (Hugo and Titanic, plus a nomination for Apollo 13) plus a couple of Emmys (Star Trek: The Next Generation and Star Trek: Deep Space Nine) under his belt, the most remarkable thing about Rob Legato's latest VFX extravaganza is that it doesn't appear to have any VFX at all! Martin Scorsese's The Wolf of Wall Street is a masterpiece of what has become known as invisible effects, and this article from our friends at Imagineer, highlighting Rob's work with mocha, is a real eye-opener.

World‐class user interface designers, graphic artists, and animators create crisp, timeless visual elements for the sci-fi film starring Tom Cruise and Morgan Freeman, Oblivion. Kosinski turned to Bradley G. Munkowitz (GMUNK), whom he'd previously worked with on Tron Legacy. As lead interface graphic designer, GMUNK then pulled a team together that included Interface Graphic Designers Joseph Chan (Chanimal) and Jake Sargeant, and Interface Animators Alexander Perry (AP), Navarro Parker (Nav), and David Lewandowski (D-Lew). They were happy to reunite to share how much fun they had creating the 2D effects for the film using an Adobe workflow.