SAVE VISUAL EFFECTS

Thursday, April 26, 2012

Beyond 24fps

Peter Jackson has been shooting The Hobbit in 3D at 48fps*. Jim Cameron is looking at doing Avatar sequels at 60fps. The preview of some of The Hobbit footage this last week caused some concern and confusion. I didn't see the footage but will describe some of the reasoning and history of why frame rates are being explored.Doug Trumbull developed the Showscan process (60fps) years ago and I worked on some projects in it as well as projected the demos for awhile. It was actually 72fps when I started but we conducted tests with people wired up and there was a definite response to the higher frame rates. Most dramatic was 24 to 48 and then it started tapering down by 60 fps. Dogs and most animals see at a higher frame rate than people. Dogs would go crazy in the showscan theater in a way they don't in a regular theater.

Keep in mind when you're seeing something projected at 24fps the projector is showing the same image 2 or 3 times (48 or 72fps) to avoid you seeing flicker because the human eye can easily see 24fps changes. On film projectors they had multi-blade shutters to accomplish this. Also note that before sound fps was 16-18 and 24fps was determined as the slowest frame rate they could use with sound. It wasn't based on research for the best quality but the least expensive to make do. Just like the layout of the qwerty keyboard (standard keyboard used around the world) was simply base on the limits of the mechanics of the day (avoiding jammed keys) and not on best usage.

Doug showed a Showscan demo that had a roller coaster ride (like the This is Cinemarama demo decades before) and a log ride. The demo would start in 35mm 24fps and then would switch to 70mm 72fps. At that point the entire audience would feel the movement of the coaster and the sloshing of the log. It was in fact a very real and visceral experience. You were there. In the log ride there was a shot of a waterfall and of a duck flying up with water drops flipping off it's wings. At 24fps that scene has a blurred waterfall and very blurred wings which we expect for film because we're used to it. A similar scene at 72fps you could see the full waterfall effect as if it were live and could see the wings of the bird just like you can in real life.

This is NOT the Saving Private Ryan look. This doesn't strobe and this isn't artificial.

One of the demos he later showed was a projectionist behind the screen leaning on the screen and talking to the audience. Most people thought it was real.

This works great for simulator rides where you have action going on. I directed the Space Race simulator ride at ILM. We shot it on VistaVision but it was created for 60fps Showscan. If you saw the results converted to 24fps compared to true 60fps there was no comparison. In one you're watching a movie and in the other you're experiencing the movie. This is why most special venue films and simulator rides run at higher frame rates since they're not locked into a standard and since they want to provide a real experience.

For film it produced a grain free experience and with 70mm it showed everything so makeup and closeups could be problematic. Just as 4k resolution, higher frame rates give more of a sense of details for flaws on set are more readily apparent. As with 3D stereo, some of the old filmmaking cheats are less tolerated.

With 3D stereo you're relying on eye disparity. And with 24fps you actually get a lot of strobing for any image that moves sideways or during a pan. And that's one of the problems Cameron and Jackson are trying to solve by going to a higher frame rate.

So with 48 or 60fps you have a much more real and visceral experience. You tend to move or jump in your seat more during action sequences or POV shots because you're mind is no longer looking at an illusion projected up on the screen. Your subconscious mind is responding because it now believes much of what it's seeing is real. The difference of going 2k to 4k is much less than going from 24fps to 48 or 60fps. You can see the difference on any device, any size. Those with newer TV sets can get a pseudo feeling for this by switching on the 120fps or 240fps cinema mode. But realize this is a simulation, not real original images, and realize it's on a small screen compared to being 40 feet or more across.

Part of the problem is video was developed to run at higher frame rates. For years of course it was inferior in terms of quality and live events have a certain look. This includes soap operas. One of the main reasons for that look was the 60 fields per second. And that's why 24fps video, when it finally came, was overwhelming chosen by filmmakers because it had a look like 'film'. It wasn't because of lack of depth of field or film grain that caused the obvious difference of video to film. It was primarily the frame rate.

Had video been at 24fps it's much more likely audiences would be more welcoming to higher frame rates. As it is anything starting to resemble live video has a stigma attached.

Because 24fps has a bit of an illusion to it and because we've been so ingrained by seeing so much of it all our lives, it provides a slightly dream like quality to this form of story telling. At higher frame rates action sequences will seem more real but likewise so will the people, acting and sets. Will this make them appear more as soap operas for standard dialog scenes? I suspect young people use to video games will have no difficulty adjusting but for most of the audience it may be a bigger jump.

24fps may be like the qwerty keyboard that continues years beyond it was required simply because it formed a standard that everyone learned and is comfortable with, despite not being the most efficient.

As far as visual effects is concerned all of these newer processes (4k resolution, 3D stereo, higher frame rate) add more work and require more time. That's why the compression of post-production schedules combined with the desire from the studios for more advanced effects work and the additional workload of these new technologies results in massive overtime and frustration of the visual effects crews.

3D stereo requires twice as much rendering and requires some tasks to be done twice. Some things that visual effects relied on, such as standard 2D matte paintings, can no longer be done in the same way.Moving from 2k to 4k resolution results in 4 times the size of image data. That takes more time to render, composite, move files, etc and takes 4 times the amount of disk space.Higher frame rates increase many of these tasks in proportion to the increase in frames. Many frame by frame hand work (and yes, believe or not people work by hand on individual movie frames) such as rotoscoping will obviously require more care. Hopefully splining and other other systems will minimize the additional frame specific work but there will always be some.

*FPS stands for Frames Per Second. These are the number of still images displayed in sequence in a second of time to create the illusion of motion.[Update 12/2/2012 Just saw THE HOBBIT at 48fps. Itdefinitely reduced the stereo horizontal strobing that's common in stereo films and smoothed some of the fast action. There were a few odd shots in regard to motion. Some seemed to have been sped up in post. A little sense of video in some of the earlier scenes but once the movie gets going it doesn't draw attention to itself but certainly added a bit more realism with the 3D and higher frame rate. I hope to see it again soon to review more fully at a close position to the screen. ]

[Update 12/13/2012 I do recommend people try to see it at 48fps with an open mind so you can decide yourself. This could be one of the only times you get the opportunity to see a full feature film at 48fps (if the results don't sit well with the studio they will not do it again) As I've covered elsewhere this of course is the first attempt to do a full film at this rate so there are bound to be issues that will have to be sorted out. Also in the future we may see variable fps films. 24fps for 'normal' scenes and then a shift to higher fps for action scenes,POV scenes or as a creative choice to different viewpoints or sequences just as directors choose at times with color grading or other manipulation. Doug's film BRAINSTORM was supposed to show the brainstorm experiences at 60fps so they were different. Wizard of Oz has black and white until the Oz sequence. ]

8 comments:

"One of the demos he later showed was a projectionist behind the screen leaning on the screen and talking to the audience. Most people thought it was real."

That's kindof the key ingredient of illusion, isn't it? Context and a bit of misdirection. It seems like when you want to introduce something new you have to slip it in under the creative radar or people will be jolted out of their suspended disbelief. We've all got a whole lifetime of mental programming that says a movie is 24fps.

My personal feeling as a VFX veteran and life-long film snob, for what it's worth, is that artistic standards need to be raised in kind with technological standards. Simply flipping a switch on more quality, more frames, stereo, CGI etc. etc. doesn't automatically make a film any better.

Ok so more frames means smoother motion, which drastically changes how shots in motion will play on screen which should completely change how those shots are composed etc etc etc...

That's a platitude and probably seems like an obvious criticism but sometimes it's worth stating. Dramatic revisions to a 100 years of cinematic visual language will require invention of new words and phrases or your audience will be unhappy with a poor translation.

1) It pulls me out of the experience. If the intent is to make a film more real and visceral, it fails on me. When I look at high frame rates, like what Michael Mann used for "Public Enemies" or the 60fps often used in video games, it's very off-putting to me. In my eyes, it doesn't look real, it looks fake. Even films converted to PAL feel odd and off-putting to me. Granted I've never seen Showscan, so I can't speak to that, but what I have seen I don't like. I know people may say I'm just a traditionalist or a film snob or "I'll just have to get used to it," but I don't want to get used to it.

Being a filmmaker who started out shooting on video is probably the main reason for this. Back before 24fps video, we tried every trick in the book to make movies shot on video look like film, from high-contrast imagery to frame blending and deinterlacing - anything we could find to get rid of that video look. So seeing people I respect advocating doing the reverse of what I strived for years to accomplish is very disheartening for me.

2) I quite simply don't want movies to become more real and more visceral. I watch a movie to escape reality, not to be sucked back into it.

3) If you need higher frame rates to create a visceral experience and pull an audience into your story, then you have failed as a storyteller.

Just my opinion, of course. People may like the new format, or at least get used to it, and I may be forced to accept it eventually, but I sure as heck am not gonna go quietly. :-)

Agreed! The 60fps experience is going to be reserved for 'shock' films such as horror, VR, games, camcorder found footage, and the like. It's just TOO close to what we see and the costumes look like costumes... that foam rock that worked in 24p looks like ass in 60fps! The makup shows in 60fps... the actors look like actors. 60 fps turns a film into a play, where you can move the audience member around in the play.. but plays have a higher suspension of disbelief than films. Just take a look at any of the blu rays being played on 4k TVs with the 'trumotion' or tweening frame speed up effect. They look AWFUL, like you are watching behind the scenes footage. Not for a second do you believe a pirate ship is a pirate ship in 60p... you KNOW it's a set. A movie at 24p is a painting, a movie at 60p is a photograph. Use accordingly.

Hmm, video games have the advantage that they calculate the motion blur. So you have the blur of 24 or 25 fps on the 60 or more fps motion. This adds to a movie look while having video framerates but is technically not possible to recreate with live action.

I managed to see the last 10 minutes or so of The Hobbit part 2 the Desolation of Smaug, in the HFR mode, and I have to say that I really did not like it. It did not look cinematic at all, it just looked really fake. The images looked sterile. Since just about all cinemas are now projecting digitally, it seems that the "problems" of 24 fps are emphasized. I am already longing for the good old days of when film was being projected.

Recent Comments

Current state of VFX life

To get up to speed on what's happening to visual effects artists and workers around the world and their troubles:
Life After PI documents one of the many closing of a visual effects company.
This is on Rhythm and Hues which worked on Life of Pi and went bankrupt a few weeks before receiving an Academy award. Life After Pi