ADAM: The evolution of Alembic support in Unity

Read how ADAM’s facial mocap and cloth simulation got into Unity, producing highly realistic CG humans, all rendered at 30 FPS in real time. To accomplish this, Neill Blomkamp and Oats Studios made great use of our Alembic Importer, Timeline, vertex sharing, and other standard features.

I’m SerEn (Sean) Low, an Art Tools Engineer specialized in Tools and Pipeline on the Made with Unity team. Prior to joining Unity Technologies, I was at Disney Interactive, developing animation tools for their console games, which collaborated with Pixar, Marvel and Lucasfilm. For the ADAM films, I helped integrate tools such as Alembic and Timeline, making sure the animation pipeline is streamlined and easy to maintain.

Background: The birth of Alembic in Unity

About two years ago, the Unity R&D team initiated the Alembic Importer plugin. It was used in a short film named The Gift by Marza Animation Planet. In one of the most challenging shots in the film, there are millions of plastic balls that form an ocean wave, which is done by importing the Alembic data generated from a Houdini simulation into Unity. It was a solid proof point that Alembic could be integrated into Unity by allowing artists to bring their simulated data from various external applications. Alembic also allows artists to break through the limitations of joint-based mesh deformation in a real-time engine.

When Oats Studios decided to use our Alembic Importer in their pipeline, they also wanted additional functionality and optimization so that it could handle high visual fidelity for cloth simulation and real-life facial-performance capture. Since the requirement was vastly different from what the Alembic Importer was originally developed for, Unity’s Dev team put in a huge effort to upgrade this plugin.

Major upgrades for the Alembic Importer

The first major update of the Alembic Importer was Timeline Integration. This allows Alembic data to stream in and playback through Timeline during runtime. Simultaneously, during edit time, the user can scrub through Timeline while viewing the Alembic playback. To enable this integration, we added a custom PlayableBehavior (Alembic Stream Player), PlayableAsset (Alembic Shot Asset) and TrackAsset (Alembic Track) to the Alembic Importer package. (For details about the Playables API, check out the documentation.)

In Timeline, you can then add an Alembic Track, create an Alembic Shot Asset and associate it with the Alembic instance in-scene with an Alembic Stream Player component attached.

An Alembic Track and Alembic Shot Asset in Timeline

An Alembic Stream Player component in scene object

The second significant update for the Importer was support for Vertex Sharing, which was important to optimize Alembic streaming performance for constant mesh topology assets. While typical Alembic data is used for dynamic simulation such as particles (where mesh topology is always changing), we wanted to merge shared vertices in order to optimize the playback performance. This is highly important since the mesh is reconstructed every frame based on the data streaming in real time. By merging the shared verts, other mesh properties such as edge, UV, normal and tangent are updated too, so the dynamic Alembic mesh now behaves like an FBX mesh. This ensures the mesh reacts according to assigned materials, shaders and lighting in a scene.

Alembic mesh in Unity showing shared vertices

For the third major update, we added Interpolation in Alembic vertex data. Since Alembic mesh is basically a new mesh constructed every frame, it doesn’t have the information about the delta of the previous and next vertex position. This is an issue when you try to apply motion blur onto Alembic mesh for rendering. With interpolation enabled, the Alembic asset in Unity is able to calculate the interpolation between positions and thus output the velocity that is necessary to calculate motion vectors. That means you can now have accurate motion vectors derived from the Alembic mesh to be used in shader computation or by the Post-Processing Stack. This enhances motion blur quality in final rendered images and it also improves performance because fewer samples are needed.

Motion vector on Alembic mesh

Before motion blur applied

After motion blur applied

The motion vector buffer

Additionally, several other production-friendly features were implemented to enable a smooth production workflow, such as auto-activate on Alembic Playable Asset, which enables the activation of the Alembic Playable Asset without the need of an additional Activation Track in Timeline. The Time Scale property allows you to re-time the playback rate of Alembic data. The Turn Quad Edge property allows you to choose which orientation the mesh faces should triangulate in. There is also an option to cache all Alembic samples whenever the Alembic scene object is enabled, instead of having to cache the data during playback.

Alembic in production at Oats Studios

“Alembic streaming was a very significant aspect of the project for us. Bringing simulated things over is difficult and the Alembic Importer in Unity is a game-changer,” said Oats Studios’ VFX Supervisor, Chris Harvey.

Assembling multiple Alembic assets in Timeline editor

Based on real wardrobes, all the clothing in Oats’ ADAM films was captured by photogrammetry and then recreated and simulated in Marvelous Designer, where digital tailor Sean Frandsen applied real physical attributes and animation to the different fabrics (wool, cotton, leather, etc.).

Meanwhile, facial performance was captured and converted into a series of static meshes for Alembic playback. (For more information about facial capture, watch the ADAM: Behind the Scenes video.)

In ADAM, Alembic is mainly used for two types of assets: Cloth and faces

The data was then imported into Autodesk Maya for further cleanup and tweaking before being exported out as an Alembic file. Here’s a schematic of their pipeline.

The Alembic and FBX assets pipeline into Unity

Assembling simulated assets in space and time

At this stage in Unity, Oats used Timeline as an assembly tool, combining Alembic assets and choreographing them in scene space along with other assets. For example, in order to fully assemble a full character performance, they used two Alembic tracks, one each for facial and cloth, on top of an Animation track, which contains body performance in FBX format as shown below.

The assembled character comes to life in Timeline

With everything in Timeline, Oats could synchronize and sequence all asset performances according to each shot’s timing for real-time playback. However, large numbers of assets can easily overcomplicate the Timeline editor interface and be hard to maintain once in production. Oats solved this by organizing assets in-scene using TrackGroup, which also let them expand or collapse groups as needed. Another Unity feature that Oats tapped to keep things well organized and manageable was ControlTrack for handling multiple nested Timeline instances.

One-click automation

Later in the production stage, the amount of generated Alembic data for both facial mocap and cloth simulation reached almost 300 files. Therefore Oats needed an automated process to import multiple Alembic files into Timeline, so they implemented their own importer to automatically bring in Alembic files with predefined settings, instantiate them into a scene with the required components as well as sequence them in Timeline. With help from the Unity Dev team, Oats was able to kick-off this automation process with the click of a button. We also gathered a lot of great feedback and API requests from this project to further improve Timeline.

Managing hundreds of Alembic assets for various characters and props

Looking ahead

Over the course of the ADAM project, the Alembic Importer became more mature and production-ready. You can download the release version at the quick link below. Now you can use it for dynamic topology simulation mesh as well as static topology mesh such as cloth and face. It’s optimized and has better integration for existing and upcoming Unity features. More and more artists and studios are showing interest in using Alembic, especially for real-time CG films, so we’re looking forward to seeing how it will enable the success of many awesome Unity projects. Let us know how you’re using it!

42 Comments

So after testing this I have to ask. Why is the version on GitHub so far behind what has been done with the Adam Series. None of these shorts could have been made with this Plug-In in it’s current state. Does Oats Studio have a Dramatically different version that we could have access to?

Hi Eddie,
We were using the vertex-sharing branch for Adam project, that’s might be why you see the main line is behind. It’s been merged now and we’re doing all necessary tests. So no, Oats doesn’t have any special build, the Alembic Importer is publicly available :)

According to Sean in the below quote Now this is integrated into Unity. But Import Asset does not even see .abc files…

“Sean Low
December 11, 2017 at 7:46 pm

Hi Narendra,
Alembic Importer is now integrated so you can go Asset>Import New Asset and point to a .abc file. You will see the abc file asset in your Asset folder :) You can then drag drop the abc into your scene. To playback the alembic:
1. Hit Play, and the alembic stream will kick in.
2. In Timeline, add Alembic track, add Alembic clip, then link the abc scene obj in the ‘Stream Player’ of Alembic Shot Asset.
Hope helps :)”

Really great tool!
I am importing alembic files to use in timeline for a 360 rendered movie. The reason for alembic is the mesh is changing poly count over the sequence. Think lava lamp / blob mesh. Some imports with a larger surface area are divided into smaller chunks on import. This works perfectly if the material is opaque but not if the material is transparent. lots of flashing faces etc. Is it posibble for somehow force the import to keep the mesh as one on import.

Can we get a bit more info on what’s being updated by the interpolator? I assume vertex data and possibly normals, but what about UV?

I ask because I have an alembic clip with a sample rate of 5 frames. When it’s animating in Unity, it loops choppy with “interpolate samples” turned off (as expected), but is smooth when “interpolate samples” is enabled. However, the shading (shadows) on it looks pretty bad, and I’m trying to figure out why and how to fix this.

Hi Sean Low,
Thanks for your tool which will open news doors for artists.
I am working on a VR project in UE4 to create a tool which will allow the artist and the customer to see the progression of the work.https://youtu.be/7qWXEylM3ns
I was working with an Htc vice but recently I acquired an Oculus Rift and start to work with Unity.
I am very dependent of Alembic as I import clothes from Marvelous.
My question is: Is it possible to import an .abc file from a folder outside the builded game?
Because as this is a tool the artist must be able to import his/her creation inside.
Thanks for your cooperation.

Hi Heurtaux,
Thanks for sharing your VR project!
You can import .abc file from any location and the Importer will create a copy in ‘StreamingAssets’ in your Unity project, to be used when you build your game.

Hi BlackPete,
Yes, Alembic Importer will create another copy in StreamingAssets which will be used when publishing the project. In ADAM films, most of the single abc file are within MB, and total abc files is less than 10 GB . If you are doing mesh deformation like a character, full body cache is not recommended. Try split it and only use Alembic where you need high detail deformation. Hope helps :)

Hi Narendra,
Alembic Importer is now integrated so you can go Asset>Import New Asset and point to a .abc file. You will see the abc file asset in your Asset folder :) You can then drag drop the abc into your scene. To playback the alembic:
1. Hit Play, and the alembic stream will kick in.
2. In Timeline, add Alembic track, add Alembic clip, then link the abc scene obj in the ‘Stream Player’ of Alembic Shot Asset.
Hope helps :)

Good News! Do you have any plans adding Android-Support for this plugin? I think for now you just can play Alembic animations on window, linux etc. Would be nice to have support for mobile devices too.

The skin shader in this project is soley on our legacy render pipeline, as ADAM EP2 & EP3 were built on 2017.1. My teammate will be posting an article about the shading part in this project soon. Please keep an eye on that!