Interior Mapping

I have come up with a nice new technique to suggest interiors in buildings: interior mapping. What the technique does, is smartly read from a ceiling-texture to show a ceiling behind windows. Here are some images to show what I mean. This is a standard box, without any internal planes. It has a diffuse texture, cubemap reflections and interior mapping.

Latest demo can be found further in this topic.

The technique requires one extra dependent read in the pixel shader and can be extended to also do interior walls and floors (right now the ceiling texture is just used as the floor as well). Doing walls, ceilings and floors would require four dependent texture reads, though, so that is more expensive. I think this technique will be especially interesting for shooters in urban areas, where most choose to make the windows simply 100% reflective. This technique can suggest a lot more depth to the graphics.

Now I have never seen this technique used before, so I think I might have come up with something New, Cool and Fresh, but I do not know whether that is actually true. So I hope you folks can tell me whether this ahs been done before.

The idea comes from an effect a friend of mine, Ralph Rademakers, once noted in a game: the texture mapping was broken due to a bug and what he saw suggested interiors. He asked me whether it would be possible to do something like that on purpose and I came up with this. The math is really simple: the ray from camera to pixel is traced further in object space to ceilings on fixed heights. The height for the next ceiling is found using the floor()-function, so this allows an infinite number of ceilings at fixed distances without extra costs for the number of ceilings.

If this has not been done before, then I will experiment further with this technique, trying optimisations and further uses of this technique.

Posted: Tue Mar 27, 2007 11:51 am

by Vectrex

that's really cool, runs nicely too.

Posted: Tue Mar 27, 2007 12:25 pm

by Thamas

Cool technique.

I do think the look is a bit messy: I'm not entirely sure I would have understood what I was looking at if you hadn' told me. (I will test it on an unsuspecting subject tonight; see what he thinks . ) A better texture would probably go a long way, but you might also consider heavy fog inside the building. (If I glark correctly how you're calculating things, this would be trivial to add.)

Posted: Tue Mar 27, 2007 1:44 pm

by Chris Jones

i like the effect, adds more depth to the building. the 1 thing i dont like about games is when you have a 3d building with a really flat texture on it which includes all the windows/doors etc.

Re: Interior mapping

Giving each floor its own ceiling would be worth it IMO. Can you give us any shots of this?

Posted: Tue Mar 27, 2007 4:39 pm

by vimes

It's an interesting feature but it might get better if you
1 - create a second texture for the floor so that it looks less confusing
2 - you activate this effect only when the camera is in a certain range of the building ( better activate it progressively if it's possible)
I would have had looked into it myself, but I'm busy trying to get the exampleapplication working

Posted: Tue Mar 27, 2007 5:37 pm

by sinbad

That's a really nice idea, haven't heard of it before.

Posted: Tue Mar 27, 2007 6:06 pm

by Aladrin

This is really good looking. You aren't gonna copyright the technique and make us all hate you, are you?

This is definitely one of the things I want to play with once I get off my fat butt and get coding.

Posted: Tue Mar 27, 2007 7:03 pm

by SpannerMan

Thats really cool, good job. Is it easy enough to configure a little, like say for example to:

a) change the amount of lighting on each level's ceiling/floor texture? For example a level may have its lights turned off.
a) use multiple textures so the different levels of the building appear to have different ceilings / floors?

Posted: Tue Mar 27, 2007 7:12 pm

by Aladrin

Unless I am badly mistaken, There is actually only1 ceiling, it just appears to have multiple. You can't vary the lights and ceilings on each floor because they don't actually exist.

Cool trick However: I think it might in practice not be that more expensive to actually put some planes inside the building and have "real" floors and ceilings. But I wonder if this technique has other interesting uses?

Posted: Tue Mar 27, 2007 7:33 pm

by Kencho

Looks interesting... Modeling large cities might make a good use of this. You can do very large batches in terms of vertex count, and still have a nice depth and detail on the buildings!

Posted: Tue Mar 27, 2007 8:54 pm

by Oogst

I went a bit further with this, now that it seems to be indeed a new technique. So far I think that creating good textures for this is pretty difficult and I will have to put more effort into that to get it to actually look good. These buildings are almost all windows, which is quite unnatural and also emphasizes the Interior Maps too much, I think, showing their seems and horizon.

I put a new version online, this time with several different materials to show a bit more of how it works, and a floor added to it. The result looks like this:

Thamas wrote:...you might also consider heavy fog inside the building. (If I glark correctly how you're calculating things, this would be trivial to add.)

Fog would seem odd, I think, but I might put a limit on the distance at which the map is shown and show some dark grey from a certain distance instead, suggesting a wall inside the building.

Aladrin wrote:...You aren't gonna copyright the technique and make us all hate you, are you? ...

Nope. I intend to write a short paper about it instead, just for the fun of it.

SpannerMan wrote:...Is it easy enough to configure a little, like say for example to:

a) change the amount of lighting on each level's ceiling/floor texture? For example a level may have its lights turned off.a) use multiple textures so the different levels of the building appear to have different ceilings / floors?

Changing the lighting per floor is actually not very difficult, because I can just calculate some random lighting value from the height of the floor. That way a specific floor will always get the same lighting and the next floor will be darker or lighter.

Multiple textures could be done as well, but the technique is pretty expensive as it is, so I do not think it would be worth it to do that. It might be interesting to put several floors in one texture, though, and do some texture coordinate jiggling to switch them per floor. Might work for little price that way, as I am already doing dependent reads anyway. I wonder how that would look, though, as a repeating pattern in switching floor textures would definitely look very ugly and I am not sure I could make it random. And even random the repeating of two is probably more ugly than of one.

Nope, that one is just about reflections in windows. Reflections are in my demo and images, though, as an added effect, but the Interior Mapping I am talking about here is the ceilings inside the building.

jjp wrote:...I think it might in practice not be that more expensive to actually put some planes inside the building and have "real" floors and ceilings. But I wonder if this technique has other interesting uses?

I think the price of the effect greatly depends on the texture resolution, as dependent reads like this will kill the texture cache, I suppose. Doing reads close to previous reads is important for performance. So experimenting with texture resolution might help performance.

As for putting floors there: this technique is especially interesting for high buildings with lots of floors. For polygons, that would mean lots of polygons, while this technique still costs the exact same performance. Also, putting planes there requires a lot of extra textured planes that need to be blended with reflecting windows in front of them. Alpha blending is pretty expensive to do. All in all, I am hoping that this is more efficient than putting actual planes there, but I will have to test that to be sure, of course.

Posted: Tue Mar 27, 2007 9:46 pm

by vimes

It's really strange : up close and with an angle the result is really convincing, but as soon as you're a bit far (i.e. when you can see the whole building), it seems wrong, somehow... Maybe it is because there is no far boundary on the and there is no wall inside, I'm not sure.

But anyway, the effect as it is now might be useful to create Escher-ish building or surrealist landscape, for those who wants to use in a unrealistic/unbelievable fashion.

You'll notice that indeed, skyscrapers that are mostly glass are designed to have their outer surfaces almost completely reflective. Draw you attention, however, to the second one. I believe you can just make out some details through those windows.

Looking at some other non-skyscraper building pictures, they are designed so you can see a bit more inside. Looking at your demo pic and then actual photos I'm not convinced you technique is flawed. It seems a combination of window size, window clutter and reflectivity may make all the difference. A good texture artist could create interior and window textures which combine to make a very convincing effect.

Keep going, I think you have something! Don't write just a short paper for fun, write a long one for fun and see what you get out of it.

Posted: Tue Mar 27, 2007 10:56 pm

by Aladrin

Oogst wrote:

Aladrin wrote:...You aren't gonna copyright the technique and make us all hate you, are you? ...

Nope. I intend to write a short paper about it instead, just for the fun of it.

I was kidding of course It would have been perfectly within your rights to license this out, instead. I'm glad you're not, of course. It's amazing.

Posted: Wed Mar 28, 2007 2:59 pm

by jomunoz

It's really cool. Doesn't give the impresion of being a cube at all.

Nice idea.

PS: The only thing i notice is that the room looks like having infinite space, i mean it doesn't have a right or left wall (depending on how you look at it.)

Posted: Sun Apr 01, 2007 1:19 pm

by Oogst

Because of the problems with the planes going to infinity, I have been working on combining several planes to get rooms instead of infinite floors. Currently the rooms do not have textures, which looks like this:

(This is still just a cube made of 12 triangles! These planes are not in actual geometry!)

I hope to have a version with some form of texturing this evening and I will put a new executable online then as well.

Posted: Sun Apr 01, 2007 1:30 pm

by Aladrin

Oogst wrote:(This is still just a cube made of 12 triangles! These planes are not in actual geometry!)

I was like "doesn't this kind of kill the original concept" but then you went and explained and made it 10x cooler again

I'm really, really looking forward to seeing that textured and shown off like the first post. Simply awesome, man.

Posted: Sun Apr 01, 2007 5:46 pm

by ahmedali

I think the technique used in this project is called Camera Mapping, commonly used in programs like 3dsmax/maya to map textures/pictures to 3d objects based on camera position and angle. If generalised, there may be many other uses too.

Posted: Sun Apr 01, 2007 6:14 pm

by Oogst

I will have a new version up and running in a moment. In the meanwhile: I am trying to get my cube reflections right, but somehow they get mixed up badly. Can someone tell me what I am doing wrong with them?

ahmedali wrote:I think the technique used in this project is called Camera Mapping, commonly used in programs like 3dsmax/maya to map textures/pictures to 3d objects based on camera position and angle. If generalised, there may be many other uses too.

As far as I know, camera mapping is something very different from what I am doing here. I do not know what it means in MAYA, but in 3D Studio MAX, camera mapping has hardly anything to do with interior mapping.

Posted: Sun Apr 01, 2007 7:08 pm

by Oogst

Woopsie, here the thing is with more room-like interiors! This is all just a 12-triangle box, so these planes still only exist in the shader math.

Rooms without textures:

Textures applied to the ceilings in the rooms:

One texture applied to all the walls in the rooms:

Funky effect on a sphere:

A lot of different settings, including the possibility to shift the floors around, rotate the buildings and change the mesh, are now in the executable:

The change since the first version of this shader is that I now not only check for horizontal planes, but also for vertical planes. This solves the problem with the planes going into infinity and looks quite nice, I think. It took me quite some effort to cramp everything in the 64 instructions that shader 2 gives me, though, but now I have one diffuse texture, one cube texture and one interior texture and lighting on the interior and that all works nicely, I think.

I think my next step will be to try and make a real street with buildings with interior mapping. I also still need to make a version with triangles instead of my shader to check the difference in performance. My shader has become quite complex, so I am not sure who will win...