Monday, August 6, 2012

Say you're making a Unity game that takes place in a large landscape dotted with windmills, and some of these windmills have tunnels that lead underground. But in Unity, the terrain collider is generated from heightmap data: it's essentially one giant bumpy plane. You can't punch holes in it.

Can we hack it to achieve the same result? Yep. Here's one way, there are two parts to it:

1) Hiding a piece of terrain geometry with a "depth mask" shader.
2) Disabling the collider so the player (or whatever) can pass through the hole, but collides with terrain other times.

If you need more detail, here's my specific implementation:
To decide what to draw and when, 3D engines draw meshes in a certain order (Unity calls this the "render queue") so it can figure out what blocks what from the camera view. This is what happens when, say, a particle system sprays steam particles into a wall, and the wall cuts off part of the sprite; in that case, the wall came earlier in the render queue, so it blocks the part of the sprite that is stuck inside the wall. Why draw something that's behind a wall?

So if we can trick the engine into thinking there's a wall, it won't draw anything behind it, even if we don't render the wall, meaning we can selectively cut out pieces of models. That's what we do with a "depth mask" -- it's an invisible wall that hides whatever is behind it.

I've also overridden my default Unity terrain shader, just to make it render later in the queue, at "Geometry+100" which is AFTER the depth mask ("Geometry+10"), and I do this because I want to cover the edges of the hole with a tunnel model ("Geometry"). So my render queue now looks like this: brick tunnel >> depth mask >> terrain.

If you don't modify the queues in the shaders, or you need a more complicated setup, then you might be better off writing a script that changes render queues on a per-object basis using Material.renderQueue. (For me, though, this setup is good enough because my terrain is simple and flat.)

Now we need a hole model. You're trying to model out the "mass of empty space" that the hole would otherwise occupy.

I used an 8-sided cylinder, just barely thicker than the terrain, so that it just barely blocks it. It's textured with a material using the depth mask shader, in the picture below, and you can see the blue skybox underneath:

Now we need to turn off the collision for our hole, so the player can fall through it. I chose to make a sphere collider trigger over the hole that disables collisions between the player and the terrain when the player is inside the trigger:
using UnityEngine;
using System.Collections;