Live Video Projection Primer

A work in progress, aimed as a useful overview of live video possibilities – a round-up of current software, hardware and techniques, and where possible, leaning towards the DIY side. A gathering spot for various links that illuminate some of the creative potential / opportunities available with video projection.

Projection Enhanced TheatreThe Ice Book – a performance blending animation, puppetry, and film to bring a pop-up book vividly to life in front of the audience.

Projection Mapping Stop Motion AnimationsLa Cena by VJ Suave / see also Homeless by VJ Suave (interview with VJ Suave) and Suburbia by Zeal – a clip was made using a combination of stop motion animation and projection mapping – the link includes a video describing his process of automating DSLR and projector synchronisation.

Planning. Every hour spent planning will be savoured later when it has saved time shooting, capturing, processing, editing, rendering, exporting, loading into VJ software etc etc. Some useful tools/ideas?

Understanding Comics by Scott McCloud is a great book for stimulating thoughts about how to pre-visualise and storyboard your ideas.
The Sour music video is an excellent example of how good planning can bring excellent results (in this case, utilising the webcams of users all over the world ).

Preparing for VJ performance? Think about variations of clips, templates, transitions, layers, masks, looping points. For live flexibility, think about modular clips that allow improvisation, adaptibility. Clips that loop nicely. Clips that layer and composite well together. Clips that can mask areas of the screen. Clips that allow dynamic transitions.

Production

There are a variety of basic editing packages out there to chop, layer and splice your videos. Dedicated compositing, motion graphics and visual effects software will likely prove very useful for video creation – from editors such as imovie and windows movie maker through to Final Cut Pro, Adobe Premiere or free software from Avid, and compositing software such as After Effects. In the 3D realm, there are options like Blender ( free ), 3DsMax, Cinema 4D and Unity Pro ( a game engine ). Increasingly though, it’s possible to generate and manipulate a lot of video material directly from within VJ software optimised to work via graphics cards ( rather than the CPU ).

The Basics

Make sure you know the end result needed for your video – and then figure out how to set that up within your software…. eg

CoGe (lots of Quartz integration, mac only)Isadora (another modular contender)Max / MSP + Jitter (Very powerful, customisable graphic programming language environment – People are excited about the MAX4Live software which integrates the sequencing possibilities of Ableton Live, and allows audio performers to tightly synchronise their custom Jitter parameters)Processing (More accessible variety of the above)VVVV (PC based patching software)Animata – Live animation software, controllable via Quartz Composer and OSC (eg control limb movements / animation components in real-time)Unity Game engine – Streamlined for game playing and the graphics card, this can also be used as a real-time 3D environment.

Hardware

Basic Technical needs : projectors, screens, cables, adaptors.

Capturing, Mixing, Controlling, Processing, Outputting.

Capturing : Increasingly, video is being captured onto cards or hard-drives within a camera – which then simply requires transfer to a computer for editing. A tape-based camera will most likely have a firewire, USB or analogue (RCA/composite ) connector, which enables capturing to your computer by plugging into the relevant input of your machine, or by using a capture card ( these can also be used to capture games / tv signals / live camera feeds etc).

HD camera considerations? Which codec does the camera record with? Quality and resolution of recording? Which lenses can it use?
Popular with VJs? DSLRs by Canon + Nikon. (Great for recording video in low-light conditions.)
Digital still cameras? Many can shoot high speed, which makes for great slow motion footage. Underwater casings are also more prevalent and cheaper for digital still cameras. Tiny portable cameras such as the GoPro record up to HD and enable dramatic action shots.

Controlling : Video software can be controlled by much more than a double clicked mouse. Using some sort of hardware controller is one of the best ways to make live video more instrument-like, responsive, versatile and fun. Mostly we’re talking about using knobs, buttons, sliders and touchscreens to control video software parameters. Most real-time video software also allows a range of easy automated processes to control parameters – eg letting audio analysis functions use audio levels to control a parameter, or setting up a range of mathematical processes which can add randomness or ‘swing’ to your video parameters.

Any USB connectable joysticks, dancemats or game controllers can be used to manipulate video parameters, using Software such as Junxion. The DIY electronics Arduino kit : “a tool for making computers that can sense and control more of the physical world than your desktop computer. Arduino can be used to develop interactive objects, taking inputs from a variety of switches or sensors, and controlling a variety of lights, motors, and other physical outputs.”

Processing

Aside from the VJ software listed above, hardware can be used for live visual effects. eg Circuit bending experiments – or professional equipment such as the (now quite dated) Korg Kaoss pad.

Outputting

Aspect Ratios? eg 4:3 or 16:9. What proportions does your final output need to be? Is your production work and compositing using this information well?
What codec do you need to export your videos as? (eg For optimum performance with best quality image, Resolume prefers the DXV codec.)

“All codecs involve a trade off between the amount of compression and the resultant quality. If you compress too much the quality loss may become intolerable.”

Although written to suit video exports for the web (ie more heavily compressed, to suit faster downloading and browser viewing), rather than video for projection, the Vimeo guide to compression / exporting explains succinctly some of the key parameters to be aware of when exporting a video.

– Projector issues – input types? (VGA / HDMI / DVI / S-Video /Composite) Do you have appropriate cables and connectors to send a video signal from your computer to the projector? What is the projector’s output resolution and aspect ratio? Lense angle? What is the projector’s throw?

Connecting to multiple screens / projectors can be done using external devices such as the Matrox Triple Head to Go card, which will divide for example – a very wide video sent from a single computer such as 2400×600 – onto 3 separate screens of 800×600. The more expensive Datapath X4 will send video to 4 separate screens, and includes a range of cropping and alignment features.

Dataton Watchout – A PC based platform for controlling video across multiple displays simultaneously.QLab – Mac based software for controlling multimedia events and installations. “QLab allows you to control audio, video, and MIDI from a single workspace.”

Motion?Video Mirror Units ( Austria ) allow video projections to move around, by controlling a motorised mirror at the very front of the projector. The mirror and motor are moved using DMX signals, the protocol used for stage lighting. A device called UDMX, allows this to happen through a computer via USB ( it’s essentially a DMX to USB convertor all wrapped up inside a cable ).

Projection Mapping:

Madmapper – built around the idea of sharing video content between applications, using a Mac OS X based framework called Syphon. ( List of tutorials for connecting various software to Madmapper. )VPT – a free multipurpose realtime projection software tool for Mac and Windows created by HC Gilje.Millumin – Mac software for ‘creating and performing audiovisual shows’. Includes 4 corner mapping for layers, mesh warping for screens and multi-display edge blending.Meshwarp Server – ‘a versatile Mapping Tool to create complex Video Projection Installations.’IR Mapio – ‘a set of editor and plug-in of FreeFrameGL standard to arrange 2D mapping projections of arbitrary complexity’ – to be used within VJ apps such as VDMX, Resolume or Modul8.

We can generate our own 2D and 3D objects for projection onto. As well as manual experiments and building, there are precision options such as laser cutting and printing 3d shapes – and a range of origami software for printing paper models with the necessary fold marks:

ORI-REVO: – A Design Tool for 3D Origami of Revolution.ORIPA – An Origami Pattern Editor.Treemaker – a program for the design of origami basesPepakura Designer – make templates for paperwork models from 3D data files.

Post-Production

Software? Adobe After Effects, Motion and many, many other applications can go beyond the edit, and provide rich processing capacity. As well as these traditional tools, many VJ applications provide capacity for real-time exploration of effects and manipulation.

Budding visual artists looking for something like Photoshop with a timeline, usually find themselves at the door of After Effects soon enough. And while higher end software packages exist, it remains a sophisticated and powerful application for creating animation or motion graphics, for layering and compositing text, graphics and video, and for adding special effects or post-producing ( colour corrections etc ). There’s an abundance of tutorial sites online, but there’s a few resources that are particularly handy :

videocopilot.net – has a great range of free introductory video tutorials, and sells more advanced tutorials ( as well as software plugins ).ae.tutsplus.com – Probably the most regularly updating AE blog out there.aescripts.com – Useful scripts that can make several tasks easier ( auto-detect edit points and separate clips into individual layers? No problem. )creativecow.net/after-effects-tutorials-podcast – The pioneering motion graphics site also has 150+ video podcast episodes to wander through, from easy through to brain-squintingly hard.