Fluffy predator with THREE.js & instanced geometry

For a recent project I had to manipulate a potentially large number of meshes. I chose to use THREE.InstancedBufferGeometry as it is very efficient ; it allows to draw the same mesh many times, with different attributes, in a single drawcall.

Working with instances is slightly different from using regular meshes. In a nutshell, you give the blueprint of the mesh you want to draw, adding attributes as you would with a THREE.Buffergeometry and set some special THREE.InstancedBufferAttribute that will affect each instance of the blueprint.

If you understand the concept of blueprint (the actual geometry), instance (the placeholder for a blueprint) and InstancedBufferAttributes (the variables applied to each instance of the blueprint), you’re good to go.

The big difference with regular meshes is that instead of setting the meshes’ translation/rotation/scale (TRS for short) directly on the mesh object, you have to either, manipulate all the instances’ BufferAttributes on the CPU then re-upload them to the GPU or perform procedural transforms directly on the vertex shader.

There is a limitation when using Instances ; in WebGL, you can’t pass more than 4 floats per attribute. This may not sound like much but it has a big impact on how you manipulate the meshes. You can’t pass a Matrix4 (4*4 floats) to transform an instance. This was a surprise for me, a Matrix4 is the way meshes’ store their transformations internally, it would be convenient to pass the elements of a matrix and use this matrix InstancedAttribute it to transform each instance of the blueprint but it is not technically feasible.

When life gives you lemons…

Therefore, the easiest way to transform an instance is to use 3 InstancedBufferAttributes to represent the TRS :

translation: a Vector3, equivalent to the mesh.position

rotation: a Vector4, equivalent to the mesh.quaternion (not the mesh.rotation!)

For the vertex shader, you can do something like this (picked from the THREE examples):

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

precision highp float;

uniform mat4 modelViewMatrix;

uniform mat4 projectionMatrix;

uniform mat3 normalMatrix;

//'blueprint' attribute

attribute vec3 position;

//instance attributes

attribute vec3 translation;

attribute vec4 rotation;

attribute vec3 scale;

// transforms the 'blueprint' geometry with instance attributes

vec3 transform(inout vec3 position,vec3T,vec4R,vec3S){

//applies the scale

position*=S;

//computes the rotation where R is a (vec4) quaternion

position+=2.0*cross(R.xyz,cross(R.xyz,position)+R.w*position);

//translates the transformed 'blueprint'

position+=T;

//return the transformed position

returnposition;

}

//re-use position for shading

varying vec3 vPos;

voidmain(){

//collects the 'blueprint' coordinates

vec3 pos=position;

//transform it

transform(pos,translation,rotation,scale);

//project to get the fragment position

gl_Position=projectionMatrix*modelViewMatrix*vec4(pos,1.0);

//just to render something :)

vPos=pos;

}

and the fragment uses the world position as the fragment color:

1

2

3

4

5

precision highp float;

varying vec3 vPos;

voidmain(){

gl_FragColor=vec4(normalize(vPos),1.);

}

With the code above, the instances should be properly positioned, rotated and scaled and the mesh should look something like this:

so with this, you can already do a lot of funny stuff, anything that needs many objects to move individually. My project was to animate a wall of panels, much like how Portal levels build up. I ended up with this mesmerizing *ahem* demo:

Then I tried to assign uvs as InstancedBufferAttributes (related to each instance) rather than BufferAttributes (related to the blueprint).

Each instance now has it’s own uvs instead of using the blueprint‘s uvs.
You may wonder how the instances are animated, that’s quite simple ; you need two sets of positions (two meshes), then create a target attribute on the blueprint geometry (as it will be the same target for all the instances) and in the vertex shader, instead of writing:

1

vec3 pos=position;

you’d write :

1

vec3 pos=mix(position,target,ratio);

where ratio is a value between 0 and 1. When the ratio is 0, the instance is in the original position, when the ratio is 1, it is in the target position, if ratio is between 0 and 1, the values are interpolated between the 2.

The ratio can be passed as an InstancedBufferAttribute so that each instance opens gradually and/or with a delay and of course, the ratio can be a noise function to get fancy animations.

It is also trivial to use a distribution object – the translation InstanceBufferAttributes are the vertices of the distribution mesh, the rotation can be computed with a lookAt() – which gives this kind of things:

Btw the background music was recorded randomly, it is “I bet you look good on the dancefloor” by the arctic monkeys :)

The next natural step for me was to try to cover a mesh with instances of triangles so that any mesh with a triangular basis would fit nicely and wrap the distribution mesh.

The source triangle should be scaled, skewed, rotated and translated so that a single matrix can transform it into any triangle in space. Little did I know that aligning a triangle to another triangle using a transform matrix requires a special type of transform called an affine transform (I couldn’t explain the difference between homothetic and affine really).

When life gives you affine transforms, run.

After creating a source unit triangle (like the one used in the basic demo), my algorithm went as follows for each face of the mesh:

find its center and compute its normal

transform an Object3D so that it is positioned and oriented like the face

use the inverse of the object’s matrix to transform the triangle’s vertices so that they lie on the ‘XY’ plane

multiply the object’s transform matrix with the 2D affine transform (the one that aligns 2 triangles)

use this matrix to transform the instance

steps 1 to 3 are really straight forward ; THREE has everything you need to do this, especially as, when using an old school THREE.Geometry, you can access the object’s faces’ list where every Face object holds the face normal. step 4 implied finding the affine transform… After 2 days of research (and headaches), I found this great JavaScript example and after some refactoring, I ended up with this glorious piece of code:

now t contains transform from our source triangle to the current face.

I thought it would work but nope! In fact, I obtained a completely valid Matrix4 but, as we need to decompose the Matrix4 into the T,R & S components to pass them as attributes to the GPU, I suspect some data, required for the affine transform, went missing. In other words, it got schwifty and the triangles wouldn’t fit.

the HI-RES mesh is fairly big (250K faces, 8Mo) so it may turn your computer into a toaster.UPDATE: I’ve added a LO-RES setting (65k faces, 2Mo ) that should work better on regular computers, click to load:

On a side note, if your mesh disappears when you zoom in, it probably gets frustrum culled, try to disable culling on the Mesh:

1

mesh.frustumCulled=false;

another reason why it would disappear is if the geometry’s bounding sphere is too small, you can (hackily) fix this by doing so:

1

2

geometry.computeBoundingSphere();//recompute the bounding sphere

geometry.boundingSphere.radius=1000;// assign an arbitrarily large radius

note that this will force the render of your mesh, use with caution :)

I can think of many silly use cases for this, instanced geometry (finally) allows control over hundreds of thousands of meshes, not particles, actual meshes! I put the files on GIT repo: https://github.com/nicoptere/FluffyPredator/

I don’t know if this TRS transform is very different from your function (apart from the fact that it adds Translation and Scale :)) or if it is more efficient (it depends on whether or not the cross product is hardware accelerated) but it’s simpler to read and write.

the thing is that it needs a Quaternion instead of Euler rotations (XYZ) and that’s a touch more complex (for people like me at least ^^)

hey,
sorry I’m not much of a youtube person… :/
so for the uvs, you need to pass a scale and offset attribute to each instance ; the scale will ‘crop’ a portion of the texture – say you have a 4 * 3 grid, your scale will be 1/4 on the X axis and 1/3 on the Y axis – and the offset will tell which cell of the ‘grid’ you want to display – again with a 4 * 3 grid, to get the bottom right corner, the offset should be 3 * 1/4 = .75 / 2 * 1/3 = .66.
then in the vertex shader, you compute the uvs like so: vUv = uv * scale + offset; and you’ll probably need to flip the Y axis so adding a vUv.y = 1. - vUv.y; could help.
I haven’t tried the code but I hope it helps :)

I might have missed something, could you ellaborate on the formula you mentioned?

and also, should I pass the blueprint geometry’s UV’s, or should I, as you mentioned in the article, create an instanced bufferattribute and assign UV coordinates by hand?

If so, I tried doing that and it worked when I assigned the array as a buffer attribute, but when I used the same array as an instanced bufferattribute, my meshes are getting different colors that are similar to the image that I use as a texture!