I recently started playing with shaders in three.js and I wanted to share some of what I’ve discovered so far. Shaders are the ‘secret sauce’ of modern graphics programming and understanding them gives you a lot of extra graphical fire-power.

For me the big obstacle to learning shaders was the lack of documentation or simple examples, so hopefully this post will be useful to others starting out. This post will focus on using pixel shaders to add post-processing effects to Three.js scenes. This post assumes you already know the basics of using Three.js.

What is a Shader?

A Shader is a piece of code that runs directly on the GPU. Most modern devices have powerful GPUs designed to handle graphics effects without taxing the CPU. This means you get a lot of graphical power essentially for free.

The big conceptual shift when considering shaders is that they run in parallel. Instead of looping sequentially through each pixel one-by-one, shaders are applied to each pixel simultaneously, thus taking advantage of the parallel architecture of the GPU.

There are 2 main types of shaders – vertex shaders and pixel shaders.

Vertex Shaders generate or modify 3D geometry by manipulating its vertices. A good example is this fireball where the vertex positions of a sphere geometry are deformed by perlin noise.

Pixel Shaders (or ‘Fragment Shaders’) modify or draw the pixels in a scene. They are used to render a 3D scene into pixels (rasterization), and also typically used to add lighting and other effects to a 3D scene.

There are 2 different kinds of pixel shaders –

Shaders that draw an image or texture directly. These allows you to draw the kind of abstract patterns seen on glsl.heroku.com. These types of shaders can be loaded into a THREE.ShaderMaterial to give cool textures to 3D objects like this example.

Shaders that modify another image or texture. These allow you to do post-processing on an existing texture, for example to add a glow or blur to a 3D scene. This second type of shader is what we will be talking about for the remainder of this post.

Pixel Shaders in Three.js

Three.js has an effects manager called EffectsComposer and many useful shaders built in. This code is not compiled into the main Three.js file, rather it is maintained separately in 2 folders in the three.js root folder:

/examples/js/postprocessing - contains the main EffectsComposer() class, and a number of ShaderPasses.

/examples/js/shaders – contains multiple individual shaders.

Unfortunately these shaders are not very well documented, so you need to dig in and test them out yourself.

First we create an EffectComposer() instance. The effect composer is used to chain together multiple shader passes by calling addPass(). Each Shader Pass applies a different effect to the scene. Order is important as each pass effects the output of the pass before. The first pass is typically the RenderPass(), which renders the 3D scene into the effect chain.

To create a shader pass we either create a ShaderPass() passing in a shader from the ‘shaders’ folder, or we can use some of the pre-built passes from the ‘postprocessing’ folder, such as BloomPass. Each Shader has a number of uniforms which are the input parameters to the shader and define the appearance of the pass. A uniform can be updated every frame, however it remains uniform across all the pixels in the pass. Browse which uniforms are available by viewing the shader JS file.

The last pass in the composer chain needs to be set to to renderToScreen. Then in the render loop, instead of calling renderer.render() you call composer.render()

That’s all you need to apply existing effects. If you want to build your own effects, continue.

GLSL Syntax

WebGL shaders are written in GLSL. The best intro to GLSL syntax I found is at Toby Schachman’s Pixel Shaders interactive tutorial. Go thru this quick tutorial first and you should get a lightbulb appearing over your head. Next take a look at the examples in his example gallery. You can live edit the code to see changes.

GLSL is written in C so get out your ‘Kernighan and Ritchie’ :). Luckily a little code goes a long way so you won’t need to write anything too verbose. The main WebGL language docs are the GLSL ES Reference Pages, which lists all the available functions. GLSL Data Types are described here. Usually Googling ‘GLSL’ and your query will give you good results.

Some GLSL Notes:

Floats always need a number after the decimal point so 1 is written as 1.0

GLSL has many useful utility functions built in such as mix() for linear interpolation and clamp() to constrain a value.

GLSL allows access to components of vectors using the letters x,y,z,w and r,g,b,a. So for a 2D coordinate vec2 you can use pos.x, pos.y. For a vec4 color you can use col.r, col.g,col.b,col.a

Brightness Shader Example

Shader code can be included in the main JS file or maintained in separate JS files. In this case the shader code is in it’s own file. We can break apart the shader code into 3 sections, the uniforms, the vertex shader and the fragment shader. For this example we can skip the vertex shader since this section remains unchanged for pixel shaders. Three.js shaders require a vertex and fragment shader even if you are only modifying one.

UNIFORMS
The “uniforms” section lists all the inputs from the main JS. Uniforms can change every frame, but remain the same between all processed pixels

tDiffuse is the texture from the previous shader. This name is always the same for three.js shaders. Type ‘t’ is a texture – essentially a 2D bitmap. tDiffuse is always passed in from the previous shader in the effect chain.

amount is a custom uniform defined for this shader. Passed in from the main JS. Type ‘f’ is a float.

FRAGMENT SHADER
The fragmentShader (pixel shader) is where the actual pixel processing occurs. First we define the variables, then we define the main() code loop.

Here you will notice one of the quirks of shaders in three.js. The shader code is written as a list of strings that are concatenated. This due to the fact that there is no agreed way to load and parse separate GLSL files. It’s not great but you get used to it pretty quick.

uniform variables are passed in from main JS. The uniforms listed here must match the uniforms in the uniforms section at the top of the file.

varying variables vary for each pixel that is processed. vUv is a 2D vector that contains the UV coordinates of the pixel being processed. UV coords go from 0 to 1. This value is always called vUv and is passed in automatically by three.js

The main() function is the code that runs on each pixel.

Line 8 gets the color of this pixel from the passed in texture (tDiffuse) and the coord of this pixel (vUv). vec4 colors are in RGBA format with values from 0 to 1, so (1.0,0.0,0.0,0.5) would be red at 50% opacity.

Line 9 sets the gl_FragColor. gl_FragColor is always the output of a pixel shader. This is where you define the color of each output pixel. In this case we simply multiply the actual pixel color by the amount to create a simple brightness effect.

Mirror Shader Example

In addition to modifying the colors of each pixel, you can also copy pixels from one area to another. As in this example Mirror Shader.

This code checks the x position of each pixel it is run on (p.x). If it’s greater than 0.5 then the pixel is on the right hand side of the screen. In this case, instead of getting the gl_FragColor in the normal way, it gets the pixel color of the pixel at position 1.0 – p.x which is the opposite side of the screen.

More Shaders!

Here’s some examples of some more advanced shaders that I built recently. View the source to see how they work.

Of course this is just scratching the surface of what you can do with shaders, but hopefully it will be enough to get some people started. If you made it this far, congratulations! Let me know how you get on in the comments.

It’s received wisdom among startup types that: “Ideas are worthless, it’s the execution that counts”. It’s become so much of a cliche that the great pop-sci middle-brow heavy-weight Malcolm Gladwell has just written a New Yorker article about it.

Now obviously there is some truth to this. It’s easy to say – “Let’s build a website exactly like Tumblr, but for recipes!” And then do nothing about it. This happens to me multiple times a day.

However there is a counter argument: without the right idea, any amount of execution is worthless.

I have personally seen it many times. Brilliant technologists and designers who build a startup around an idea that just doesn’t make much sense. It either doesn’t fulfill a need, or it cannot be explained in one sentence, or it is just too similar to ideas many other startups are executing on at the same time.

So yes, the execution is incredibly important – but so is finding the right idea to execute on. Building a technically or aesthetically wonderful product is a waste of time if nobody wants to use it.

The hard part is figuring out which are the good ideas and which are the bad. Doing this requires a combination of intuition, research and general knowledge of the world around you.

Flickr founder Caterina Fake voiced a similar opinion a while ago that resonated with me:

“Much more important than working hard is knowing how to find the right thing to work on. Paying attention to what is going on in the world. Seeing patterns. Being able to read what people want.”

The purpose of this meetup is to discuss interactive 3D graphics in the browser, using WebGL, Stage3D and even Unity3D. Bartek Drozdz and I put the meetup together so that we can all learn from each other and make some good connections. These technologies came out very recently and there is a lot of potential to make some very cool stuff.

For the first meetup, I gave a presentation on the current state of interactive web 3D. View the demos here. View the slides from the talk here (PDF).

Bartek gave a presentation on his J3D WebGL 3D engine. View the slides from the talk here (PDF).

We plan on doing the next meetup sometime in January. If you are in the area and want to learn about next-generation 3D in the browser, or give a presentation – please join us.

Trying to find that cool URL that you posted to Twitter a few months ago? Want to easily archive all your tweets? Now you can with AllMyTweets. AllMyTweets displays all your tweets on one page. Just enter your Twitter user name. All of the data is pulled from Twitter’s public API, so no authentication is required. To archive your tweets, select all the text on the page, copy it and paste to a text document – old school style

Note that Twitter limits the maximum number of tweets returned to 3,200, so if you have more tweets than that you are out of luck. Also the Twitter API is not always completely reliable so you may find the loading hangs sometimes. If this happens, please try again later.

I built this because there I couldn’t find anything similar on the web, and I figured other people must want to be able to do the same thing. Please put any feedback in the comments. Thanks!

ImageGlitcher allows you to glitch any image and save the result. Glitched images look like they are being displayed on a broken analog TV. Check it out here.

This demo was built using Peter Nitsch’s BitmapData.js – a very useful JS port of ActionScript’s BitmapData class which makes it easy to manipulate canvas image data. After playing around with it for a while I got some pleasing glitched image results, so I figured it would be useful to wrap it up as a little app. The glitch effect is achieved by:

Slicing the image horizontally and randomly offsetting the slices

Randomly selecting a color channel and offsetting again

Brightening the image

Adding a ‘scan lines’ overlay.

All the source code is viewable via ‘View Source’. Surprisingly, cross browser development wasn’t too much of a pain once I ignored IE. One of the cool features of HTML5 is native drag and drop support. On Chrome and Firefox you can drag in an image from the desktop.

One issue that came up was that the canvas tag does not allow you to getImageData() on images that are from a different web domain. This is supposedly a security feature to prevent hackers copying images, although I fail to understand why. If a hacker has the URL of an image on your website he already has a local copy of the image – right? Anyway $.getImageData gets around this feature by routing image data through Google App Engine using JSONP.

If I get time there are a few features I want to add in the next version:

Add ‘Instagram’ style filters

Add more granular filter options

Use web workers to avoid hang time

Let me know if you find any issues or if you create any nice images, post a link in the comments.

UPDATED – 9 May 2013

Removed loading remote images via $.getImageData. Their service appears to be down.

With all the FUD around Flash and the iPad its easy to forget that we are living in a golden age of web development. The web is fulfilling all but the most outlandish predictions of its success from back in the 90s. Things are just starting to get interesting.

The other take-away I got from the I/O keynote is how a lot of HTML5 is basically HTML playing catchup with Flash. Many of the coolest new features have been available in Flash for years. This means all the skills you honed as a Flash developer are directly applicable to the newly emerging platforms of HTML5, Unity, Android and even iApps: