The second problem is that in the worst case, my adaptive supersampling code uses more samples than garden variety supersampling. The problem is that pixel corner samples are shared, but the recursive subsamples are not. Pixels share several subsamples along their borders.

I only have two possibly impractical ideas: Render the entire image by quadrisection instead of just the subsamples, but that would limit you to power of two image sizes. The other idea is to use bisection, which would allow you to use any image size, but I can't think of how you could deal with the subsamples in any useful manner.

Ok, I take back what I said about complete quadrisection/bisection. It would have exactly the same problem. Subdivisions have the same edge sharing problem. I was thinking about a compression project that I did where data wasn't shared at all between recursive bisection calls.

However, I found that computing the average and then comparing the distance of individual colors to the average to be ever so slightly faster. I also tried sampling at the pixel center and using that instead of the average. Surprisingly, for most moderately complex shaders it doesn't seem to make much of a speed hit, and it provides better quality at lower sampling rates.

I supposed I didn't mention that part. I've been playing around with procedural texture generation. Unfortunately I haven't been keeping most of the images, so I don't have many good examples. The first is the contours in a perlin noise function. The second is a slightly more practical example, shaded hexagon tiles.

I think I might try sampling the lower left corner of each pixel, and comparing adjacent pixels. That's pretty similar to what I was doing before, but wouldn't have the edge sharing issues.

I'm doing it like you would do procedural textures in a ray tracing program. You build a function that you can sample at arbitrary points to get the color. I just leave out the ray-tracing part and sample the texture functions directly.

The hexagon tiles function goes something like the following: A height map is built using the hexagon and perlin functions. The hexagon function returns distance from center, distance from edge, and cell index. I clamp the distance to edge value to bevel the edges of the tiles, and add some perlin noise to the top. This height map is then lit using diffuse lighting. Finally, I darken the tiles based on the hexagon index value as it's pseudo-random. This sounds complicated, but it ends up only being 10 significant lines of C code.

It was just meant to be a fun experiment at first, so it's not really that generic yet. For this to be really useful, I either need to make a simple shading language that you can define the texture functions in or create some sort of GUI editor like this.

The cool thing about procedural textures is that you can easily create arbitrarily large textures (and their associated normal and specular maps) from a fairly tiny texture description.

pretty pictures! if you're thinking about schemes to eliminate jaggies, there are at least two alternatives to explicitly supersampling (which can be expensive, even if you do it adaptively):

first, as a nice hack, you can use the smoothstep function, which gives you anti-aliasing "for free": imagine you'd like to render a bunch of polka dots, each of size 5 pixels. one function you could use is to look at the distance of any pixel's (x,y) coordinate from the center of the nearest polka dot:

Code:

dx = x - ctrx;
dy = y - ctry;
d = sqrt(dx*dx + dy*dy)

if (d <= 5) {
color = 0.0;
} else {
color = 1.0;
}

instead, using the smoothstep function, you could get

Code:

color = smoothstep(5.0, 5.25, d);

which would make pixels along the edges of the circles look nice and gray instead of jaggy.

alternatively, you can do something called "frequency clamping": if the function you are plotting has a spatial frequency on the order of 1 pixel or smaller, you can just output a mean value over the pixel (i.e. rendering lots of sub-pixel black and white stripes right next to each other should just produce a gray tone). GLSL gives you the functions dFdx, dFdy, and fwidth to estimate the coverage in texture space of a pixel in screen space.

There's an entire chapter dedicated to this topic in the Orange Book. The math is over my head, but I found it fascinating.

EDIT: Regarding the use of smoothstep -- it only produces visibly appealing results at specifically tuned sampling frequencies. Too high and you're back to jaggies, and too low and you have blurry output.

EDIT2: I'm an idiot. It would probably work fine in the example above.

You can actually download their texture editor from somewhere, Windows only though. ProFX is another good example of a commercial procedural texturing tool.

I also found a similar open source tool on sourceforge. FXGen is modeled after the Fairbauch (spelling?) tool. It's planned to be cross platform using SDL. The GUI looks to be mostly textfield driven though.