One of the ways in which Mango facilitates the understanding of how people work and collaborate with each other, is by introducing the concept of “resources” and “resource types”.

resources are the different files that are made by the pipeline(or artist in the pipeline) and handed off between different pipeline steps. One of the key designs ideas of mango is to categorize and label the different kind of resources that are created in production to better understand the following questions…

What is the main purpose of the resource?

What step of the pipeline is best suited to create this resource?

How is this resource used by artist and producers (The Pipeline)?

How do we ingest this resource into different steps of the pipeline efficiently?

How do we keep every instance of this resource used through out the pipeline up to date?

this is illustrated in the following graph. (right click on the image and choose “view image” to see a larger version)

In this graph, we illustrate the relationship between different “resource types” in the lighting to composting pipeline. in this snippet of the pipeline the following 4 types of resources are used

WorkFiles

3d Elements

preComp

Paint

comp

in this scenario, work-files mostly deal with application specific files (.max, .ma, .mb, .nk). it is important to understand that in the mango pipeline “work files” in their entirety are not designed to be handed of to the next pipeline step (dough they certainly can be). instead the goal is to have a modular system, where the artist can build “work file” with everything that they need to get the job done, but only hand off the smaller chunk of work that his responsible for via the publish process. the next person in the pipe then collects all the available resources, built by the people before him/her, and and assembles a complete file that is then used to generate new or updated resources.

In this example the lighter makes a “work file” where he/she will load or create all the resources that he need to create the “3dElements” (image sequences) that will be assembled by the comp artist.

the roto paint artist will make a “work file” where he/she will paint out markers from a plate and render this clean “paint” elements so they can be used by the compositor

the compositor will make a work file in which he will composite all the 3dElements, Paint, and PreComp images sequences and render a final comp.

in this workflow, no artist really cares or needs an other artist’s “work file”. They only need to get the resources created by “work files”.

in the case of the other 3 resource types used (3dElements, preComp, paint, comp) they all share one thing in common, they are all images sequences. But just having a resource type called “image sequence” doesn’t tells enough about how this resource is to be used in our pipeline. By breaking it down into the the following different types, it becomes more clear as what the purpose of each image sequence is meant to do, and how it is supposed to be used by the next step in the pipeline.

Why not have different resource types for each application?the reason why all this different software formats are grouped together as one resource type is because they have a very particular purpose, and that is to create other resources. no other resources in our pipeline has the ability to create new resources other than work files. so we give them they own special category.

it it possible what so ever, that this file types will be used by future resource types, in wich this files will be used to hand over data needed for some step of the pipeline, but in this case the files would never be opened and worked on.

it’s also important to note, that by making the work-file resource type generic. we can more rapidly add new software package to the pipeline… seance of all the tools dealing with work-files won’t have to be updated to account for a new resource type.

with that said Mango does track what application made the work file. and as to stream line the process of finding workfiles that belong to a particular software.

Unique Storage location per resource typeone of the benefits of splitting resources into this “purpose descriptive” types, Is that it allows us to split the storage across multiple drives or servers, An example of this can be storing all image sequences of type 3dElements on drive x:, while storing all images sequence of type comp in Y:. A studio has the option of using as many different storage server as there are resource types.

Expandability

One of the main benefits of building a pipeline with the concept of resource and resource types, centered around the idea of “Work files” that out put axillary “resources”, is that growing the pipeline to cover new techniques and technology becomes much easier, by simply introducing new resource types, and rules that govern how this new resource is going to be handed off.

this means additions can be made to the pipeline with our affecting any all-ready existing methods and practices. or having to re-engineered major parts of code to account for new on though of practices.

I’m happy to share with you guys a screen grab of the dependency viewer for “mango version viewer”.. view the versions associated with any element being reviewed any time any place with out out having to open the work-files them self’s
dependencies are hierarchical so that you can easily view them in a natural way…

Some time has passed since my last Mango related post. Thankfully, I have made significant progress. I’m glad to announce that Mango Phase 1 will be completed in the next few weeks. There are three important developments which I will address in this post: resource-tracking, the Software Launcher, and user applications and tools (stand-alone and software-integrated).

Mango’s Resource-Tracking

Mango Phase 1 has been about setting up the structure on which the entire pipeline is based. I implemented base APIs that will support all of the core stand-alone applications and software-integrated tools. The two most significant APIs in this stage of development are:

versionTracker (API for versioning of resources with database backend)

namingAPI (API for generating the folder structure that host the files that make up the files associated with versions made by versionTracker)

These two main components come together to create a resource-tracking system. In Phase 1, the development of the resource-tracking system has been focused on the needs of smaller studios in rapid production environments. This focus allows smaller studios to start using resource-tracking without being overly intrusive or creating a bottleneck for the artist by over-departmentalizing the process. To achieve this implementation, Mango Phase 1 concentrated on establishing the standards and practices necessary for creating and tracking the main resource types that are the most important in the “generalist” environment. In particular, I have focused on artist “work files”, “render elements”, and “shot plates.” In Phase 2, I will move toward more detailed resource-tracking and implement new resource types for “models”, “rigs”, “animation caches”, “cameras,” and “shaders.”

Mango’s Software Launcher

The Software Launcher is the common entrance to the pipeline. It is a stand-alone application that allows supervisors to control the software configuration by creating unique profiles for their projects. These profiles control which software, plug-ins, third party tools, and extensions are used by artists on their projects. Profiles allow artist to jump between projects, while guaranteeing that every artist under the same project has the same set of tools. Moreover, this profile mechanism allows supervisors to adopt new software or plugin versions without unintended negative effects on concurrent projects. The Software Launcher and its profile mechanisms means that studios no longer have to rely on IT to push plug-in installations over large networks via more traditional time-consuming means that can be unreliable (e.g., MSI, Psexec, manual installations).

The connection between Mango and the artist’s software relies on the modular plug-in distribution system of the Software Launcher. Mango and its tools get attached to the target application via the “LosPipeLine” Software Launcher module which should be part of every Software Launcher profile. The Software Launcher also offers and optional command line mode. This mode is used to apply the profile-based software configuration approach to network rendering solutions, such as Thinkbox’s Deadline.

User Applications ( stand alone and software-integrated)

Additionally, the deployment of Phase 1 focused on the “hand off” of resources across the different disciplines of the shot creation process (e.g., how a lighter hands off lighting elements to a compositor). Mango is composed of several applications and tools, both stand-alone and software-integrated. These tools are easy-to-use interfaces for artists, supervisors, and managers to facilitate the adoption of the Mango workflow. Here’s a list of the initial set of applications and tools that are bundled with Mango Phase 1:

Project Manager (stand-alone)

Simple interface for quickly creating the base directory for a new project

Allows for adding sequences and shots at anytime to any project

Workspace Manager (Integrated with 3ds max, maya, and nuke)

Simple interface for creating and navigating artists’ work areas, work files, and snap shots

What is PTS?

A PTS (Production Tracking System) is a database that manages VFX/Animation projects and studio resources. This system helps producers, supervisors, and artists keep track of the multiple tasks that are required to successfully complete a show. It facilitates communication between departments and provides real-time reports on the current progress and cost of work. The PTS represents the integration of a “back-end” database (e.g., postgreSQL or MySQL) and “front-end” applications. The everyday user interacts with “front-end” applications which are designed to meet his unique needs. These applications can be stand-alone or integrated into third party commercial or proprietary software (e.g., task management application or resource publishing tools inside of Maya, Nuke, etc.).

Why is there a need for a PTS?

A well-designed PTS is integral to an effective studio. It allows artists, supervisors, and producers to find relevant information quickly. For example, an artist can see a list of incomplete tasks that are assigned to him. Then, a supervisor can use PTS to comment on the tasks which artists marked “ready for review” and keep a historical log of these comments. While a producer can use PTS to see which shots are over-budget and the human resources department can have a record of billable hours that are connected to specific tasks. Everyone involved in the production process is able to input and query this information through applications that are stand-alone and/or integrated with third-party commercial or proprietary software.

A few examples of common stand-alone applications in PTS are:

A Time-Card Application

creates an electronic record of billable hours in the database which replaces paper time-cards.

allows artists to connect these hours to a particular task which can be used to create more detailed reports.

A Task Management Application

allows supervisors to break large scale work into smaller easily-delegated tasks.

holds other important information to facilitate time-management (e.g., priorities and due-dates).

connects tasks to progress-tracking statuses (e.g., “in progress” or “awaiting supervisor review) which can be filtered by supervisors and artists.

A Dailies Application

lists all the image sequences sent by artists to be reviewed in dailies.

uses the versioning system in PTS to help supervisors quickly find older versions of the same sequence for comparison during review.

connects comments about the work being reviewed in dailies to the task that generated it to ensure the artist has a record of the notes.

A few examples of common applications which are integrated into third-party software in PTS are:

Publishing and Versioning Tools

help artists version their work while keeping a historical record of changes.

allow for easier hand-off of versioned work to the next step in the pipeline.

when necessary, allow artists to easily roll-back to previous versions of their work.

adds a layer of programmable quality control to ensure smooth ingestion into the pipeline.

allows lighters and effects artists to quickly know the resources that make-up a 3d scene.

allows artists to compare version information about the resources currently in the scene.

Alerts artists when scene resources need to be updated.

Scene Building Tools

Eliminates the need for multiple artists to work on the same file by allowing artists to build unique files with shared resources.

A PTS adds accountability and power to a studio because it automates some of the more tedious aspects of the VFX/Animation pipeline. A pipeline that revolves around a well-designed PTS allows artists to focus the majority of their time on art, rather than addressing technical complications that arise from keeping scene files updated. In addition, it gives supervisors and producers the real-time information that allows them to track the progress of work and make better choices which maximize the project’s profit.

Why is a PTS not as Common in Smaller Studios?

If a PTS is so wonderful, why is everyone not using one? There are four simple answers to this question:

Smaller studios don’t usually have the resources to employ full-time programers. So, developing and integrating a PTS into their pipeline is not cost-effective.

The culture of many smaller studios means that there are no uniform standards and practices about how scene files are created and shared. Therefore, their database (the “back-end” of a PTS) mirrors this complexity and quickly outlives its usefulness.

Popular expensive commercial solutions (e.g., Shotgun/Tank, FileMaker Pro, SharePoint) are a framework for a database but, they don’t offer uniform standards and practices. This leads to the previously described problem for studios who do not have strong standards and practices. Moreover, many of these commercial solutions don’t offer third-party integrated applications leaving studios to write their own.

These expensive commercial options still require the additional cost of Technical Directors to update, maintain, populate, and further develop these systems; sometimes greatly increasing the cost of integrating such a system.

While smaller studios have a very large incentive to invest in a pipeline that revolves around a well-designed PTS, they often don’t. The expense in time and money for an inadequate product is too much of a risk.

What is Mango?

Mango is an economical plug-and-play solution that is designed around how people actually work in VFX/Animation studios. It recognizes that many studios do not have set standards and practices which are a predictable and repeatable way of creating quality work (Modular Asset Pipeline Flow). Mango not only provides a uniform way of accomplishing this work, but it is a ready-to-use PTS that is modeled around a uniform set of standards and practices. In addition, Mango includes tools that will run on multiple applications (e.g., Maya, 3ds Max, Nuke, Houdini) to help enforce the uniform standards and practices without limiting artists creative ability.

Mango eliminates the need to have multiple programmers to mold the system to work in your studio. It greatly simplifies the nature of the database that you will use to track this work. Mango minimizes the footprint that these kind of systems can have in a studio pipeline. It fundamentally changes the way the studio works. In other words, Mango helps to create and monitor a more efficient studio work system by providing a full end-to-end pipeline.

When will Mango be Complete?

Mango is an on-going personal project on which I am working full-time from home. It is the result of 10 years of experience working in a range of studios in the VFX/Animation industry. The base for the PTS and several applications will be complete by the end of July. The development cycle is going to be rapid this is why I am researching tying together multiple open-source libraries and technologies (e.g., Python, postgreSQL, SQLAlchemy). Not only will this make my development cycle more efficient, but it also makes it more accessible to other programmers.

Anyway stay tuned, I will add more posts related to my findings about database programing for the VFX industry.

One of the main interests in my career has been developing a way in which I could optimize the process of creating and revising animated content in 3d through uniform standards and practices. My interest in this topic is rooted in my experience as a Character Technical Director (TD). A Character TD collaborates with many different departments. For example, he is the one that makes sure that models are technically sound for rigging. He also has to collaborate with animators to ensure that the animation set-ups are simple yet powerful enough for the animator to be able to achieve the desired performance. A Character TD interacts with lighting and effects to make sure that the animated performance transfers from animation department to the lighting department without breaking. He is usually in charge of adding more animated detailed to assets via simulations like hair and cloth. These are just a few of the responsibilities of a Character TD,basically, he has his hands in everything. Owing to their work responsibilities, most Character TDs frequently assume some of the responsibilities of pipeline development.

While I’ve been fortunate enough to work at studios that have had well-designed asset-based pipelines, I have worked at others that have failed to develop the standards and practices necessary to implement such a pipeline. I have drawn on these professional experiences in both kinds of studios to develop a specialization in writing 3D pipelines and developing this, often missing, set of standards and practices. The link below is an image of an up-to-date concept map of how assets should flow and can be developed through a pipeline. There are many benefits to implementing this kind of pipeline. A few of the most notable include:

Modeling, rigging, shading, lighting, effects, and animation can evolve parallel to each other rather than being linearly dependent. This monumentally increases the speed with which people can accomplish work by eliminating a known bottleneck.

It facilitates the tracking of different versions of a resource that form an asset and even allow roll-back. This ensures that artists always have the correct versions of resources on which their work depends.

It eliminates the need of transitioning from previs to postvis work. This avoids a common redundant step in many small studio pipelines.

This standardized asset flow allows for a very simple database model which is able to track and manage the work greatly simplifying the development of a PTS system (mango pts).

So recently I’ve been working on maya to max pipeline…
i ran into some issues with the built in cacheFile mechanism in maya so i decided that seance .pc2 is such a simple binary format that i could just write my own .pc2 exporter!

here’s what i came up with 🙂

ps. this heavily borrowed from the blender importer written by “Matt Ebb” http://mattebb.com

Today i started to do some online research on what are the current practice for script/and shelf deployment in maya.
the studio I’m currently working for is not a “Maya” house, but we’re getting to the point where often writing tools to convert scenes animated in maya, to something we can use in max. being that this is a somewhat esoteric topic i did not expect that my internet search would return much, but to my surprise i found this article by fellow TD Jonas Avrin.

he really does a great job at introducing the concept of the userSetup.mel as a ways of deploying his personal script from gig to gig.

which seems to me would also make a great starting point along with the mayaEnv.mel to starting the journey into the full time time management of Maya at the studio level.

this might not be anything new to maya TD’s but it was very informative to me 🙂