Menu

Pixel Bender + Video = Killer Runtime Effects

I’ve been playing with Pixel Bender a bit and noticed that there weren’t really any examples (I could find) that illustrated effects being applied to video. So I gathered together a whole slew of kernels from the interweb, downloaded a copy of Dancing Matt and shoehorned them into a media player that allows you to select and apply the effects during video playback.

Instructions: Press the “play” button, then select the effect you’d like to see applied to the video.

The ten thousand dollar question is what’s the value outside of gratuitous eye candy? It seems that in some cases you might want to apply a particular stylistic treatment (wouldn’t it be great if the cartoon effect that ships with After Effects would compile for Flash Player). There’s also the opportunity to animate the filters and create some really killer transition effects–anyone with a media player component or video editing / mashup RIA ought to see the potential there. And that’s just the tip of the iceberg, I’m sure there are far grander ideas being born as I write this.

21 thoughts on “Pixel Bender + Video = Killer Runtime Effects”

One of the challenges with the real-time BixelBender effects and video is the overall processor load. For me, it was no issue because I am sitting on a 8-core box, but even then I watched the load jump to nearly 18% across all cores. Imagine trying to deploy an app to a large audience and how the user experience would be. Probably not pretty.

What I do see is some interesting prospects for media creation and real-time editing solutions. One could be an AIR app that allows people to edit video, apply filters and transitions and then export the FLV (or AfterEffects.com). Another example would be for VJs, who want to apply real-time mixing for shows and/or installations. For them, they understand that the system will put a load on their box… much smaller and controlled market.

As machines get more powerful, I can see this in more populace based applications but for now I feel the cost may outweigh the benefit of just rendering the effect and distributing the results.

@James -God point. Performance is definitely an issue and I wouldn’t recommend just applying wholesale effects during the entirety of video. I could see, short transitions (pre-roll, post-roll) with filters that have been optimized. It also seems like you could do some very fun stuff with interactions (roll overs, etc.) and thumbnails.

Hopefully round 2 of pixel bender will see the processing offloaded to the GPU whenever possible.

Awesome stuff. I particularly like the tube vision and focusing linear blur, since they have definite useful functionality imo.

I’m sure someone will answer that 10k question. There’s always creative uses for things that seem gratuitous at first. And when the tech demos are shown, people groan because they think it’s gonna be 1995 with the scrolling marquee and flashing text all over again. But then someone strolls in months later and creates something absolutely badass with the effects everyone laughed about, then people start to copy that brilliant person, then standards are created.

People raised eyebrows at all the pixel shaders and 3d effects that WPF can do, for instance, but then some people created absolutely badass and unquestionably useful UI effects with them and the laughter subsided.

Very interesting question about how we are going to use it. One thing I know is the scale line effect can definitely be used for playing full screen video. It greatly increases the video quality, in a cheating way.

Let’s wait for some genius to find out how to use some other in a genius way, then we just need to kick ourselves “why didn’t I think of it in that way?”

It would indeed be great if you’d share your code, as it’s a hassle wading through pixel benders only to find that they’re broken, no preview is available, they don’t compile, or whatever. Having a bunch of them that are labeled, preview-able and known to work would be really useful.

So we don’t really need the source for your media player, but a zip of the filters you used would prevent a lot of hunting around on the net. Hope you’ll do it! Thanks…

@Scuty / Howdy – I definitely don’t have a problem sharing my code, but in this case the Pixel Bender kernels are not my code and I just don’t have the time to gather permissions from all of the various authors. When I get a chance I’ll gather up the compiled kernels and drop a link here–hopefully that doesn’t put me at odds with the authors of the kernels).

Disclosure

This is the blog of Brooks Andrus. Here, At irregular intervals, you may find digital noise centered around the activities of an early 21st century technologist. I work for TechSmith Corporation, but this web space and the views found on it are entirely my own.