User:Nicholasbishop/Ptex

From BlenderWiki

Ptex

Get the code

Note: this code is not officially part of Blender yet. It exists only as an experimental branch (but unless something goes very wrong it will eventually get merged into master (but not before it's ready, hopefully)).

Current code is in branch cycles-ptex-49. I've been doing a fair amount of rebasing, but the code is now ready for some initial review so will probably be a merge workflow from now on. Code review: https://developer.blender.org/D1071

Recommend using CMake to build it (I did update for SCons too but haven't tested in a while.)

Quick Usage instructions

0. Starting from the default cube...

1. Set the rendering engine to Cycles Renderer

2. Set the material to use nodes

3. Add a Ptex Texture node and connect its color output to the Diffuse color input

4. From the Mesh properties panel, add a Ptex layer. Set the Ptex node's layer to the new layer.

5. Change viewport shading mode to Material. You should now be able to see the Ptex layer and texture paint on it. You can also render it with Cycles.

Changing Resolution

Beneath the Ptex layer list are two buttons, Halve Resolution and Double Resolution. These operate on selected polygons, and only on the selected Ptex layer. In Texture Paint mode the face selection mode can be used to select faces.

External Ptex files

TODO: import as layer and regular image node

How to report bugs

Once the branch is in master we'll use the regular bug tracker, but for now you can use one of these methods:

Implementation

Reference

CustomData

Internal Ptex layers as stored as loop data. Within a layer, all loops have the same data type and number of channels. Each loop has its own resolution, however.

This loop data is intended to live only in the base mesh. The modifier stack interpolates coordinates (again, loop data) that refer into the Ptex data as needed. Note that these coordinates contain an ID in addition to the UV pair. The ID is essentially the original index of the loop which contains the real Ptex data.

There is one more customdata layer that contains the ID/UVs for tessellated faces. This enables painting and rendering.

Subsurf

The Ptex spec defines two mesh modes, quad and triangles. The quad format assigns N Ptex subfaces to each NGon except for quads, which get a single Ptex face. I haven't looked at the triangle format much, the quad format seems right for our use case.

Ptex subfaces map well to storage in loops. For consistency and simplicity the Blender implementation doesn't treat quads specially for internal Ptex data. Standard Ptex quads can be trivially mapped by splitting the Ptex face into four quadrants and assigning them to the four loops of a quad in Blender.

To make rendering and painting work, the mesh must be split into quads using the same tessellation pattern as Catmull-Clark subdivision. If the mesh has a subsurf modifier at the end of its stack this already done; for other cases the DerivedMesh is automatically split with a fake subsurf (using the Simple mode so that vertices are not smoothed).

Packing

The Ptex file format is intended for use with CPU rendering. It has good support for caching and texture filtering. This should be useful for Cycles OSL rendering, and possibly Cycles SVM rendering on the CPU, but not so easy to map to GPU rendering.

As a workaround, the Ptex face data can be packed into a 2D image. This is relatively easy to do since all Ptex data is rectangles. Taken to the extreme this could enable very nice filtering if each face had lots of adjacent pixels copied around it, and duplicated for multiple mip-map levels, etc.

For now I'm taking a less advanced approach that enables only bilinear filtering (which is what Cycles uses on the GPU for other textures anyway). For bilinear filtering, only a one-pixel border of adjacent data has to be added around each region. (I'm also not including any mipmaps). Some additional metadata is required to map from faces into the packed texture.

OIIO

Currently using OIIO for all Ptex loading. Doesn't support writing yet I think, we can probably contribute patches for this. Ptex adjacency data isn't properly exported by the OIIO API yet, but working around it by using mesh metadata from the file: http://ptex.us/metakeys.html

Ptex library

Since OIIO has some support for Ptex, I hope to avoid using the Ptex library directly from Blender. For the initial release OIIO has everything we need, but if we want to add support for better filtering (which at least for OSL we definitely should) OIIO's Ptex support will need to be improved. I'm sure they would welcome patches for that.

BPX

Some of the code is written in C++. Doesn't have a fully defined interface yet, currently living in extern/ptex since that's where I had imported the "real" ptex library (before switching to just use OIIO).

Does reasonable things in edit mode (probably just free/copy/clear, ignore interp for now)

Nearest and bilinear filtering

Saving/loading

uint8 & float32 layers

Halve/double resolution of selected faces

Exclude

OSL rendering of internal Ptex layers

OSL rendering from .ptx files

Ptex as input to modifiers (e.g. displace modifier)

Filtering better than bilinear

uint16 & float16 layers

Support for more than four channels of data

Automatic texel allocation based on face sizes

Ptex "triangle" mesh format (NGons are handled well in the "quad" format)

Export to Ptex files

Baking

Conversion between Ptex and regular UV texture maps

TODO

Finalize nodes

Things to consider: Cycles, GLSL, from-layer, from-file, SVM+OSL

Texpaint issue: (Not specific to Ptex, but more of an issue due to quad requirement): model as drawn has different quad splits from the ones Blender chooses, so texpainting doesn't quite line up in the viewport (or in the render). I assume that the mesh is being drawn with OpenGL quads and so OpenGL gets to choose the split, which is bad.