Wednesday, 7 December 2011

I've just started looking at the Alembic interchange format which looks like a really promising system for passing baked animation data plus a whole lot more around. With big players like ILM, Sony Image Works, The Foundry etc all using it I thought it would be good to get started with implementing my own wrapper for the ngl:: library and just have a general look at what it can do so I can do a lecture or two on it.

Installation is fairly straight forward, you need boost++, HDF5 and cmake and the OpenEXR framework libraries ( ILMBase which has IMath etc). On the mac the latest version get placed in /usr/local/alembic-1.0.3 and this will have the static libs, headers and maya plugins needed to get started.

I'm not going to dwell too long on the structures etc of Alembic at present and just have a look at getting started on some very basic loading of meshes etc. A good overview can be found here.

Getting some geometry

To get some sample geometry to test I decided to generate a very simple scene of primitives in maya

I have named each of the primitives to I can see how they fit into the alembic structure, also as transforms are present within the alembic structure, I decided to duplicate and transform some of the elements to see how this worked.

To export the data we can use the maya exported which comes with Alembic. This is called AbcExport and can be run from the Mel script window. I've not really managed to figure out how selection / grouping works with the exporter as yet, and it seems best to make each individual element in the scene a root and export in turn. This can be done with the following command.

You will notice that we need to include both the hdf5 and ILM libs as well as the static alembic libraries.

Basic File read / echo

The first program I wrote was a basic traversal of the structure looking for nodes / meshes. From initial reading of the docs It seems that the internal structure of an alembic file closely resembles as unix file system. With a root node / then nodes following down a tree structure. Each of these nodes is a distinct object that we can access and gather attribute values from.

In most of the sample code / demos a recursive visitor pattern seems to have been used for initial test I wanted a simple iterative solution for quick testing and debugging so I decided to use static loops instead.

One of the key design philosophies behind Alembic is that the API is split into In and Out classes similar to iostream. This is summed up well by the following quote in the documents

"Because Alembic is intended as a caching system, and not a live scenegraph, the API is split into two roughly symmetrical halves: one for writing, and one for reading. At the Abc and AbcGeom layers, classes that start with ‘O’ are for writing (or “output”), and classes that start with ‘I’ are for reading (or “input”). This is analogous to the C++ iostreams conceptual separation of istreams and ostreams."

So in the above we are opening an input (I) archive.

The next section of code will get the top of the archive and see how many children there are

Once we have the number of children we can iterate for each child in the node and traverse the tree (it must be noted in this example I know the tree is of a set depth, in reality we would need to traverse using recursion / visitor pattern to cope with more complex scene / geometry data however for a proof of concept this is fine)

The code above grabs the child header of the current object and prints out the name. We then create a new IObject from the current object branch (getChildHeader(i).getName() ), this is the next object in the tree and we can then traverse this.

To access all of the data we can grab the meta data and call the serialize method, followed by traversing the data to see if we have a mesh and seeing what the name is. The output on the scene above looks like the following ( a partial listing )

Getting to the Points
Once I have the ability to grab the mesh in the file structure I can now look at accessing the point data and rendering it. This is a two stage process with the maya output as we have a top level transform node and then the point data. We need to get the transform matrix, then we need to multiply the points by it to get them in the correct world space.

Using the previous code frame work, we can grab the transform using the following

You will see that this returns a M44d which is from the IMath library and is a 4x4 Matrix. We can also access the XformSample elements in a number of ways such as getTranslation, get[X/Y/Z]Rotation, getScale which will actually fit in well with the ngl::Transform class once I get to that stage. Initially I'm just going to use the raw matrix and use IMath to do the multiplication with the point.

We now do the check to see if we have a mesh as outlined above, If we do we can built a MeshObject and access the data as show below

// we have a mesh so build a mesh object
IPolyMesh mesh(child,child2.getName());
// grab the schema for the object
IPolyMeshSchema schema=mesh.getSchema();
// now grab the time sample data for the object
IPolyMeshSchema::Sample mesh_samp;
schema.get( mesh_samp );
// get how many points (positions) there are
uint32_t size=mesh_samp.getPositions()->size();

Another important concept of Alembic is the idea of sampling, again for the documentation

"Because Alembic is a sampling system, and does not natively store things like animation curves or natively provide interpolated values, there is a rich system for recording and recovering the time information associated with the data stored in it. In the classes AbcA::TimeSamplingType and AbcA::TimeSampling, you will find that interface."

We access the mesh sample at the current time value (0 as we've not set any frame data yet) and we can get the positions (and size) at this temporal sample.

The following code will now loop for each of the positions in the sample and multiply them by the current transform. I then save each in my own format Vec3

for(uint32_t m=0; m<size; ++m)
{
// get the current point
V3f p= mesh_samp.getPositions()->get()[m];

// multiply by transform
p=p*mat;
// store for later use
data.push_back(ngl::Vec3(p.x,p.y,p.z));
}

In the case of this demo I just create a point cloud into a Vertex Array object and draw using my ngl:: framework. This can be seen in the following video

A lot of this post is supposition / basic rantings about what I've done so far, as this is a very new API and there is very little solid documentation at the moment, I think this post may well be updated / superseded very soon. It is my intention to fully integrate the mesh and possibly lights / cameras as much as I can into ngl, and we also intend to use this as a core data exchange format at the NCCA, so I hope to have much more detail about all this very soon.

Thursday, 3 November 2011

I'm just about to put the feature freezes on NGL 4.5 and this blog post is going to highlight the many changes from version 4.0 which many of you may have used in last years assignments.

The Big Picture
Perhaps the biggest change in the whole library is the conversion OpenGL 3.2 core profile only. This design decision was made for a number of reason, primarily for speed and compatibility across platforms, but also so I can teach the very latest versions of OpenGL and CGI techniques. The only major issue you may encounter with this change is that only Mac OSX Lion is now supported.

The main reason for this decision was so we could use GLSL 150 which is only available under mac using Lion and also makes it easier to port all the code base to OpenGL ES ( there is an iPhone port of NGL which will be released soon). If you don't wish to upgrade to Lion then you will have to use the legacy version or develop under another platform.

This also means that all immediate mode OpenGL has gone, (under Lion they are not present as it uses two different libraries for GL2.0 and GL3.2 see here ) so it means you can no longer use functions like glBegin, glVertex, glRotate, glTranslate, glScale etc. It also means that the whole OpenGL lighting and Material subsystem has also gone and we need to use shaders to do everything. I've already mentioned some of these things in previous posts.

It has now also been tested to work with boost 1.4.7 and the Windows version seems to be a lot more stable (but will still be the last version to be developed as I hardly ever use windows)

Changes
The following list shows the major changes to NGL from 4.0 to 4.5

ngl::Clip class has been removed as the student version of Houdini no longer support ASCII clip export, this has been replaced with the NCCAPointBake system and eventually Alembic support will be added

ngl::ShaderManager has been replaced with ngl::ShaderLib, this is mainly a name change after some healthy in class discussions with Ian about class naming. Also all calls to the shader subsystem no longer need the name of the shader passed as a parameter. It is assumed that the currently active shader is the one to set parameters for.

ngl::VBOPrimitives has been replaced with ngl::VAOPrimitives, the class is effectively the same, however we now use Vertex Array Objects to store our primitives, this also has advantages for porting to OpenGL ES 2.0

ngl::VertexArrayObject class was added in 4.0 but is now used internally to create all geometry required in any of the classes. This has made the removal of any immediate mode OpenGL much easier.

ADDLARGEMODELS define has been added to the NGL.pro file, if defined it will add 4 new primitives to the VAOPrimitives class these are the NCCA Troll, Stanford Bunny, Buddha and Dragon. These models are very large and should only be enabled if required. The image below shows them in action on the iPhone 4s

New Classes

The following new classes have been added, some are to help with GLSL compatibility and some because of the changes to core profile required it.

ngl::Mat3x3 a simple 3x3 matrix class which can be created from the ngl::Matrix class. This is generally used to calculate the normal matrix for GLSL shaders and lighting calculations.

ngl::Vec3 is a simple container for a GLSL vec3 type, it is basically a subset of the ngl::Vector class and is used internally for passing vertices etc to the GLSL

ngl::Vec2 is a simple container for a GLSL vec2 type and works similarly to above.

ngl::Text this class is described in detail here but is needed as we can no longer use the Qt renderText method as it relies on immediate mode OpenGL

Demo Programs

I'm currently in the process of converting the demo programs to work on the new Version of NGL and this is taking a little time as there are quite a lot of changes in the way the shaders work and this does take some time.

Thursday, 20 October 2011

I've been working on porting the ngl:: library to be completely OpenGL core profile 3.2. One of the main issues I've had is the rendering of text. In the last version I used the QGLWidget::renderText method, however this is not available when using core profile OpenGL and also has a habit of crashing when using certain modern OpenGL elements.

To overcome this problem I have designed and implemented my own OpenGL text renderer and the following post will explain the process / design of this and my approach. The approach I use follows the standard methods used in other packages and the following survey gives a good overview of these. My main concern with the design was that it only uses Fonts from Qt and that it works with OpenGL 3.2 core profile and above.

Initial Process
The usual process of text rendering is to create a billboard for each character in the text string and apply a texture to that quad for the given character. The image below shows a basic set of billboards and a (not to scale) set of text glyphs

The process we need to follow to create the text to render is as follows

Load a font and generate all the characters required

Generate OpenGL textures for each individual character

Generate billboards for each glyph depending upon the char height and width

Store this data for later text rendering

As the design of this system evolved It was decided to foce the user of the fonts to decide in advance the size and emphasis of the the font. This was done so the texture / billboard generation only happens once at the start of the program, however once this is done any text can be rendered using this font set. This makes any programs using the text much quicker as the data is effectively cached as OpenGL textures / Vertex Array Objects

Loading the Font

The initial design decision was to create a single texture image containing all the visible ASCII characters, however this was soon dropped as there are issues with the ways that fonts are constructed to have different kerning values. So for the final design an individual image / texture is created for each font, with the character calculated to be at an origin of (0,0) Top left of the screen. Using this we will need to then calculate the height of the font once and for each character generated store the width of the different text types.

As we are using Qt we can use the QFont class and the QFontMetrics class to gather all of this information. For each class the initial design was to store the following :-

This class stores the individual width of each character's glyph gathered from the method QFontMetric::width() as well as the id for a generated texture and a pointer to a vertex array object which will contain the Vertices and UV data for the billboard.

The following code shows the basic process of generating each of the font textures.

// so first we grab the font metric of the font being used
QFontMetrics metric(_f);
// this allows us to get the height which should be the same for all
// fonts of the same class as this is the total glyph height
float fontHeight=metric.height();
// loop for all basic keyboard chars we will use space to ~
// should really change this to unicode at some stage
const static char startChar=' ';
const static char endChar='~';
// Most OpenGL cards need textures to be in powers of 2 (128x512 1024X1024 etc etc) so
// to be safe we will conform to this and calculate the nearest power of 2 for the glyph height
// we will do the same for each width of the font below
int heightPow2=nearestPowerOfTwo(fontHeight);
// we are now going to create a texture / billboard for each font
// they will be the same height but will possibly have different widths
for(char c=startChar; c<=endChar; ++c)
{
QChar ch(c);
FontChar fc;
// get the width of the font and calculate the ^2 size
int width=metric.width(c);
int widthPow2=nearestPowerOfTwo(width);
// now we set the texture co-ords for our quad it is a simple
// triangle billboard with tex-cords as shown
// s0/t0 ---- s1,t0
// |\ |
// | \|
// s0,t1 ---- s1,t1
// each quad will have the same s0 and the range s0-s1 == 0.0 -> 1.0
ngl::Real s0=0.0;
// we now need to scale the tex cord to it ranges from 0-1 based on the coverage
// of the glyph and not the power of 2 texture size. This will ensure that kerns
// / ligatures match
ngl::Real s1=width*1.0/widthPow2;
// t0 will always be the same
ngl::Real t0=0.0;
// this will scale the height so we only get coverage of the glyph as above
ngl::Real t1=metric.height()*-1.0/heightPow2;
// we need to store the font width for later drawing
fc.width=width;
// now we will create a QImage to store the texture, basically we are going to draw
// into the qimage then save this in OpenGL format and load as a texture.
// This is relativly quick but should be done as early as possible for max performance when drawing
QImage finalImage(nearestPowerOfTwo(width),nearestPowerOfTwo(fontHeight),QImage::Format_ARGB32);
// set the background for transparent so we can avoid any areas which don't have text in them
finalImage.fill(Qt::transparent);
// we now use the QPainter class to draw into the image and create our billboards
QPainter painter;
painter.begin(&finalImage);
// try and use high quality text rendering (works well on the mac not as good on linux)
painter.setRenderHints(QPainter::HighQualityAntialiasing
| QPainter::TextAntialiasing);
// set the font to draw with
painter.setFont(_f);
// we set the glyph to be drawn in black the shader will override the actual colour later
// see TextShader.h in src/shaders/
painter.setPen(Qt::black);
// finally we draw the text to the Image
painter.drawText(0, metric.ascent(), QString(c));
painter.end();

First we need to create a QImage to render the character to a glyph. As most OpenGL implementations required power of 2 textures we also need to round our image size to the nearest power of two. To do this I found a handy function here which is called with the image width and height to make sure we get the correct texture sizes.

Next we set the texture fill to be Qt::transparent and the image pen colour to be Qt::black. As the actual text rendering is done by a special shader this value is unimportant, what we are actually after is the alpha values in the image which are used to indicate the "coverage" of the text (more of this later).
After this is done we use the QPainter class to render the text into our QImage.

This stage is all Qt specific, as long as you have a Font library which allows you to save the glyphs into and image format and gather the font metrics it should be easy to port to other libraries.

Generating the Billboards
Now we have the font dimensions and the glyph as a QImage we can generate both the billboard and the OpenGL textures.

To generate the textures we use the following code

// now we create the OpenGL texture ID and bind to make it active
glGenTextures(1, &fc.textureID);
glBindTexture(GL_TEXTURE_2D, fc.textureID);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
// QImage has a method to convert itself to a format suitable for OpenGL
// we call this and then load to OpenGL
finalImage = QGLWidget::convertToGLFormat(finalImage);
// the image in in RGBA format and unsigned byte load it ready for later
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, finalImage.width(), finalImage.height(),
0, GL_RGBA, GL_UNSIGNED_BYTE, finalImage.bits());

The image above show the original sketch for the Billboard as two triangles, with the UV cords and the sequence of generation. The ngl library has a simple class for storing this kind of data called the VertexArrayObject all we have to do is pass the data to the class and then use it to draw. The following code does this.

// this structure is used by the VAO to store the data to be uploaded
// for drawing the quad
struct textVertData
{
ngl::Real x;
ngl::Real y;
ngl::Real u;
ngl::Real v;
};
// we are creating a billboard with two triangles so we only need the
// 6 verts, (could use index and save some space but shouldn't be too much of an
// issue
textVertData d[6];
// load values for triangle 1
d[0].x=0;
d[0].y=0;
d[0].u=s0;
d[0].v=t0;
d[1].x=fc.width;
d[1].y=0;
d[1].u=s1;
d[1].v=t0;
d[2].x=0;
d[2].y=fontHeight;
d[2].u=s0;
d[2].v=t1;
// load values for triangle two
d[3].x=0;
d[3].y=0+fontHeight;
d[3].u=s0;
d[3].v=t1;
d[4].x=fc.width;
d[4].y=0;
d[4].u=s1;
d[4].v=t0;
d[5].x=fc.width;
d[5].y=fontHeight;
d[5].u=s1;
d[5].v=t1;
// now we create a VAO to store the data
ngl::VertexArrayObject *vao=ngl::VertexArrayObject::createVOA(GL_TRIANGLES);
// bind it so we can set values
vao->bind();
// set the vertex data (2 for x,y 2 for u,v)
vao->setData(6*sizeof(textVertData),d[0].x);
// now we set the attribute pointer to be 0 (as this matches vertIn in our shader)
vao->setVertexAttributePointer(0,2,GL_FLOAT,sizeof(textVertData),0);
// We can now create another set of data (which will be added to the VAO)
// in this case the UV co-ords
// now we set this as the 2nd attribute pointer (1) to match inUV in the shader
vao->setVertexAttributePointer(1,2,GL_FLOAT,sizeof(textVertData),2);
// say how many indecis to be rendered
vao->setNumIndices(6);
// now unbind
vao->unbind();
// store the vao pointer for later use in the draw method
fc.vao=vao;
// finally add the element to the map, this must be the last
// thing we do
m_characters[c]=fc;
}

Text rendering
To render the text we need to convert from screen space (where top left is 0,0) to OpenGL NDC space, effectively we need to create an Orthographic project for our billboard to place it on the correct place.

The following diagrams show the initial designs for this.

As each of the billboards is initially calculated with the top right at (0,0) this transformation can be simplified. The following vertex shader is used to position the billboard vertices.

This shader is passed the scaleX and scaleY values from the Text class these values are calculated in the method setScreenSize as shown below.

void Text::setScreenSize(
int _w,
int _h
)
{
float scaleX=2.0/_w;
float scaleY=-2.0/_h;
// in shader we do the following code to transform from
// x,y to NDC
// gl_Position=vec4( ((xpos+inVert.x)*scaleX)-1,((ypos+inVert.y)*scaleY)+1.0,0.0,1.0); "
// so all we need to do is calculate the scale above and pass to shader every time the
// screen dimensions change
ngl::ShaderLib *shader=ngl::ShaderLib::instance();
(*shader)["nglTextShader"]->use();
shader->setShaderParam1f("scaleX",scaleX);
shader->setShaderParam1f("scaleY",scaleY);
}

This method must be called every time the screen changes size so the x,y position of the font are correctly calculated.

The xpos / ypos uniforms are the x,y co-ordinates of the current text to be rendered and the actual billboard vertices are passed to the shader as the inVert attribute. This is shown in the renderText method below.

void Text::renderText(
float _x,
float _y,
const QString &text
) const
{
// make sure we are in texture unit 0 as this is what the
// shader expects
glActiveTexture(0);
// grab an instance of the shader manager
ngl::ShaderLib *shader=ngl::ShaderLib::instance();
// use the built in text rendering shader
(*shader)["nglTextShader"]->use();
// the y pos will always be the same so set it once for each
// string we are rendering
shader->setShaderParam1f("ypos",_y);
// now enable blending and disable depth sorting so the font renders
// correctly
glEnable(GL_BLEND);
glDisable(GL_DEPTH_TEST);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
// now loop for each of the char and draw our billboard
unsigned int textLength=text.length();
for (unsigned int i = 0; i < textLength; ++i)
{
// set the shader x position this will change each time
// we render a glyph by the width of the char
shader->setShaderParam1f("xpos",_x);
// so find the FontChar data for our current char
FontChar f = m_characters[text[i].toAscii()];
// bind the pre-generated texture
glBindTexture(GL_TEXTURE_2D, f.textureID);
// bind the vao
f.vao->bind();
// draw
f.vao->draw();
// now unbind the vao
f.vao->unbind();
// finally move to the next glyph x position by incrementing
// by the width of the char just drawn
_x+=f.width;
}
// finally disable the blend and re-enable depth sort
glDisable(GL_BLEND);
glEnable(GL_DEPTH_TEST);
}

Before we can render the text we need to enable the OpenGL alpha blending and we use the SRC_ALPHA, ONE_MINUS_SRC_ALPHA so the text will be rendered over the top of any other geometry. I also disable depth sorting so if the text is rendered last it should always appear over any geometry.

This shader is quite simple, it takes the input texture (the glyph) and grabs the alpha channel (which we can think of as the coverage of the ink). We then set the rest of the colour to be the user defined current colour and this will get rendered to the screen as shown in the following screen shot

Future Work

As an initial proof of concept / working version this works quite well. It is reasonably fast and most fonts I have tried seem to work ok (including Comic Sans!)

The next tests / optimisations are to determine if we have any billboards of the same size and only store ones we need. This should same VAO space and make things a little more efficient.

Update

I've actually added code to do the billboard optimisation, all it needed was

QHash <int,vertexarrayobject * > widthVAO;
......
// see if we have a Billboard of this width already
if (!widthVAO.contains(width))
{
// do the billboard vao creation
}
else
{
fc.vao=widthVAO[width];
}

For Times font we now only create 15 Billboards, for Arial 16 unique billboards, for Courier only one as it's a mono spaced font.

Thursday, 6 October 2011

Term has started and a number of students have been asking about installing the version of Ubuntu we use in the labs on their home machines. There are many ways of doing this and my preferred way is to delete windows and only use Ubuntu, however most students don't want to do this as they want windows for gaming etc.

The simplest solution to run a basic linux box suitable for basic programming / exercises needed in the first year of our courses is to use Oracle (Sun) Virtual box and install a basic linux system.

Getting Started
First we need to download a couple of packages. The first is the virtual box host software which can be found here and a linux iso ready to install. For the current university build we are using Ubuntu 10.4 LTS which can be found here and we are using the 64 bit version.

The following videos show the install process and how to proceed.
Part 1

Thursday, 22 September 2011

For the first time in quite a few years I'm teaching first year programming, and had to re-visit the classic hello world program. I've decided to upload a few video walkthroughs of how to write the hello world program in python, C and C++ so I don't have to repeat myself too much.

Tuesday, 28 June 2011

I'm currently writing a model viewer which allows the user to load meshes and textures to see how things will look in a realtime engine.

One of the elements I'm currently working on is the lighting system. The main plan is to have very basic 3 point lighting and give the user some control over the lighting system and the ability to set the colours etc.

I designed a basic panel to start testing this as shown here

Signals and Slots

Usually we would associate a series of signals and slots with each of the components to allow for interactions. For example if the x double spinbox is changed the valueChanged(double) signal is emitted and we can connect it to a slot (method) to process it. In the case of this first panel I would have called the same slot for each of the signals of the widgets as the light class would have to be updated with all the values.

This multiple widget -> single slot would work fine for one panel, however as you see above I have three different light groups.

I could have 3 sets of different slots for each of the different light controls, however there would be a lot of duplicated code and maintenance would have been difficult, also at some stage I may wish to dynamically add lights so again the code would be redundant.

Dynamic Properties

What I need to do is to have some easy way to determine which widget belongs to which Light group so I can process them based on the actual group.

To do this we can use the Qt Dynamic properties tab in designer.

Clicking on the + tab in the properties inspector above give the option to add a String or Bool (or other) dynamic property. In this case I'm going to add a property called LightName and will set it to the name of the light panel the component belongs to (for example FillLight).

Once the property has been added you should have a new element of the property panel as shown below

Accessing Dynamic Properties

Now we have associated the property to the components we need to access them. First we connect each of the widgets to a method

As you can see each of the widgets is connected to a single method called updateLight() in this method we are going to determine which widget called the method and determine it's dynamic property name.

First we use the sender method of the QObject class to determine which object sent the signal. Once we have the object pointer we can call the property method to access the dynamic property "LightName". You will notice that the property method returns a QVariant object so we need to convert it to the correct data type (in this case a QString) to process it.

For now I'm just printing out the different light group values, but eventually this will be used to change all the light properties.

Wednesday, 18 May 2011

#!/usr/bin/python
import os
def scanForFiles(_dir) :
# grab all the files in the directory
files = os.listdir(_dir)
# create an empty dictionary for the results
result = {}
# this will hold our sorted list to be returned
sortedList = []
# these are the valid file extensions as a tuple as ends with needs it
validFiles=("tiff","tif","exr","jpg","jpeg")
# process each of the files found and see if it's in the format
# name.#####.ext
for file in files:
# so first we try and split on the . and see if we have a valid suffix
try:
prefix, frame, suffix = file.split('.')
if suffix in validFiles :
try:
# if we do we add it to the dictionary (using the frame value)
result[prefix][0].append(frame)
except KeyError:
# we have a new file sequence, so create a new key:value pair
result[prefix] = [[frame],suffix]
except ValueError:
print "ignoring file in directory %s" %(file)
# now we have a list of data I want to return in the form that nuke needs
# in this case we need name.%xd.ext so we grab the data and format the string
for file in result :
# get the filename
fileName=file
# get the extention
extention=result[file][1]
# grab the numeric start and end values
minFrameValue=int(min(result[file][0]))
maxFrameValue=int(max(result[file][0]))
# this is a bit of a hack as the values could be different
# but we basically form the frame pad values
framePadSize=len(result[file][0][0])
padString="%%0%dd" %(framePadSize)
fileNameString="%s.%s.%s" %(fileName,padString,extention)
# add the data to the list
sortedList.append([fileNameString,minFrameValue,maxFrameValue])
# sort it and return
sortedList.sort()
return sortedList
path = nuke.getFilename("Select Directory")
files=scanForFiles(path)
print files
for fileData in files :
fileName=fileData[0].split(".")
fileName=fileName[0]
try :
fileNode = nuke.nodes.Read(file="%s/%s" %(path,fileData[0]),name=fileName)
fileNode.knob("first").setValue(fileData[1])
fileNode.knob("last").setValue(fileData[2])
fileNode.knob("on_error").setValue(3)
Shuffle = nuke.nodes.Shuffle(inputs=[fileNode],name=fileName)
except :
print "error on ",fileName

Wednesday, 6 April 2011

Been using some maya and python and discovered there was no way to access some of the MImage methods as they use pass by reference to return values. This after some investigation lead me to the MScriptUtil class.

Now a couple of things struck me when reading the documents, the first being

"This class is cumbersome to use but it provides a way of building parameters and accessing return values for methods which would normally not be scriptable".

To my mind this reads as

"This is a hack but we couldn't be arsed to re-factor loads of our code to add proper accesors and mutators"

On further reading you get even more of a sense of that. What I wanted to do was to access the width and height of an MImage and access the RGB(A) data of the MImage and the only way of really doing this within python is to use the MScriptUtil Class.

MImage Class DesignI decided to wrap up all the functionality I needed into a simple python class which could be re-used within maya, the basic class design is as follows

The class will be constructed with a filename and will automatically read the image file and grab the image dimensions as well as a pointer to the data.

import maya.OpenMaya as om
import sys
class MayaImage :
""" The main class, needs to be constructed with a filename """
def __init__(self,filename) :
""" constructor pass in the name of the file to load (absolute file name with path) """
# create an MImage object
self.image=om.MImage()
# read from file MImage should handle errors for us so no need to check
self.image.readFromFile(filename)
# as the MImage class is a wrapper to the C++ module we need to access data
# as pointers, to do this use the MScritUtil helpers
self.scriptUtilWidth = om.MScriptUtil()
self.scriptUtilHeight = om.MScriptUtil()
# first we create a pointer to an unsigned in for width and height
widthPtr = self.scriptUtilWidth.asUintPtr()
heightPtr = self.scriptUtilHeight.asUintPtr()
# now we set the values to 0 for each
self.scriptUtilWidth.setUint( widthPtr, 0 )
self.scriptUtilHeight.setUint( heightPtr, 0 )
# now we call the MImage getSize method which needs the params passed as pointers
# as it uses a pass by reference
self.image.getSize( widthPtr, heightPtr )
# once we get these values we need to convert them to int so use the helpers
self.m_width = self.scriptUtilWidth.getUint(widthPtr)
self.m_height = self.scriptUtilHeight.getUint(heightPtr)
# now we grab the pixel data and store
self.charPixelPtr = self.image.pixels()
# query to see if it's an RGB or RGBA image, this will be True or False
self.m_hasAlpha=self.image.isRGBA()
# if we are doing RGB we step into the image array in 3's
# data is always packed as RGBA even if no alpha present
self.imgStep=4
# finally create an empty script util and a pointer to the function
# getUcharArrayItem function for speed
scriptUtil = om.MScriptUtil()
self.getUcharArrayItem=scriptUtil.getUcharArrayItem

Initially the class was designed to check to see if alpha was present and determine if the data was packed as RGB or RGBA and step through the packed data accordingly, however on further reading of the documents I discovered

"The image is stored as an uncompressed array of pixels, that can be read and manipulated directly. For simplicity, the pixels are stored in a RGBA format (4 bytes per pixel)".

So this was not required and was removed.

Using MScriptUtil
There are several ways to use MScriptUtil, the various constructors allow us to generate an object by passing in an object as a reference value, or we can create an instance of the class and then associate an object as a reference.

Initially I generated one MScriptUtil class, and associated the pointers from an already instantiated object. However this didn't work correctly. After reading the help I found the following note

So if you need to use two pointers at the same time you need to create two MScriptUtil objects.

self.scriptUtilWidth = om.MScriptUtil()
self.scriptUtilHeight = om.MScriptUtil()
# first we create a pointer to an unsigned in for width and height
widthPtr = self.scriptUtilWidth.asUintPtr()
heightPtr = self.scriptUtilHeight.asUintPtr()

Next we set the values to 0 for both the pointers, whilst this is not required, it make sure when we load the actual values if nothing is returned we have a null value.

Next a call to the MImage getSize method is called

As you can see in this method both the width and the height values are passed by reference. The code below passes the new pointers we have created using the script util class into the getSize method and these will be associated.

See all this code could be avoided if MImage had getWidth and getHeigh accessor methods!Now for some speedups

To access the pixel data we need to grab the array of data from the MImage class, this is done with the following method call

self.charPixelPtr = self.image.pixels()

The help says that

"Returns a pointer to the first pixel of the uncompressed pixels array. This array is tightly packed, of size (width * height * depth * sizeof( float)) bytes".

So we also have a pointer that we need to convert into a python data type. This can be done with the getUcharArrayItem however this would need to be created each time we try to access the data.

In python however it is possible to create a reference to a function / method in the actual code. This is done by assigning a variable name to a function and then using this instead of the actual function call. This can significantly speed up methods as the python interpretor doesn't have to lookup the method each time.

The following code shows this and the pointer to the method is stored as part of the class

getPixelsTo get the pixel data we need to calculate the index into the pointer array (1D) as a 2D x,y co-ordinate. This is a simple calculation as follows

index=(y*self.m_width*4)+x*4

In this case we hard code the 4 as the MImage help states that the data is always stored as RGBA if this were not the case we would have to query if the data contained an alpha channel and make the step 3 or 4 depending upon this.

The complete method is as follows

def getPixel(self,x,y) :
""" get the pixel data at x,y and return a 3/4 tuple depending upon type """
# check the bounds to make sure we are in the correct area
if x<0 or x>self.m_width :
print "error x out of bounds\n"
return
if y<0 or y>self.m_height :
print "error y our of bounds\n"
return
# now calculate the index into the 1D array of data
index=(y*self.m_width*4)+x*4
# grab the pixels
red = self.getUcharArrayItem(self.charPixelPtr,index)
green = self.getUcharArrayItem(self.charPixelPtr,index+1)
blue = self.getUcharArrayItem(self.charPixelPtr,index+2)
alpha=self.getUcharArrayItem(self.charPixelPtr,index+3)
return (red,green,blue,alpha)

As you can see the index is calculated then the method saved earlier is called to grab the actual value at the index location (Red) then index+1 (green) index+2 (blue) and index+3 (alpha). For convenience I also wrote a getRGB method as shown

def getRGB(self,x,y) :
r,g,b,a=getPixel(x,y)
return (r,g,b)

Other methods shown below are also added to the class to allow access to the attributes, whilst python allows access to these class attributes directly, when porting C++ code usually we will have getWidth / getHight style methods so I just added them.

Using the Class
The following example prompts the used for a file name then loads the image, anything in the red channel of the image with a pixel value greater than 10 is used to generate a cube of height r/10

Friday, 25 March 2011

My usual day to day development environment is a mac, and I usually write code using Qt / g++ so it's easy to port it to Linux which is the main development system at the University. However most of the students seem to use Windows as their main computer system so I had to really support windows with my NGL library.

Luckily the Windows version of the QtSDK comes with a g++ compatible compiler called mingw and once the other libraries were installed in the correct places the library seems to compile and link into a DLL.

Problems with GLEW

The first problem I encountered was with the GLEW library. The versions available for download are compiled with the visual studio compiler and are not compatible with the mingw compiler / linker. To make this easier to use and integrate into the ngl library I added the source of the latest GLEW tree to the ngl project and this is included into the windows branch of ngl automatically in the Qt project file. This is done using the following code

Once this code is in place, the windows version of Qt will set the win32 specific flags and compile correctly and also include the glew elements statically into my library and hence reducing a dll dependancy.

Cross Platform Development
Some other conditional code was also required to make things compile correctly on all platforms. NGL uses 3 flags to set specific platform dependent includes and flags. These are found in the .pro file as part of the qmake process and the following shows these

So not the compiler will pass on the command line -D[DARWIN][LINUX][WIN32] dependent upon the platform being used. This is then used to switch in the main header files for the libs, and to place this all in one include file the <ngl/Types.h> is used and has the following conditional compilation code

With all this in place the library built with no problems and a .dll / .a file was generated under windows and a simple demo application would run. I left the library under windows at this stage and continued the development under mac, however I began to get reports of things not working under windows. Initially I thought it was a driver / gpu problem as it seem to occur in the shader manager class.

When things go wrong

The first indications of things not working was when a lookup to a shader returned the null program which is a safety mechanism. The following code shows the method and the call to the method.

It appeared at first that the map find method was at fault as it was not finding a shader that I knew was in the class. However I started to print out the size and contents of the map I noticed that in some cases the map contained the correct values and in other cases it was empty. Why would this be?

This shouldn't happen!

It seemed weird that this was happening as the class was based on a singleton pattern so there should only be one instance of the class, also this bug only appeared in the windows version so I was unsure what was going on.

I placed a number of debug statements in the singleton template class used for the shader manager and discovered that the constructor was being called twice WTF!

My initial response was that I was having a weird threading issue and my singleton wasn't thread safe, so I added a QMutex / QMutexLocker construct around the singleton and also made it inherit from boost::noncopyable so that it could not be copied.

This however was not the solution to the problem as it still continued.

Digging Deeper

After digging deeper into the code I traced exactly where the rouge call to the singleton ctor came from. In my TransformStack there is a convenience method to load the current Transform into a parameter of the shader, the code is actually in the ngl::Transform class and looks like this.

This method actually lives in the library, and hence under windows in the DLL, and this is where the problem begins.

Separate compilation units
It seems in the windows DLL the library data is in a different scope when constructed unlike the unix / mac shared libraries. In this case when the dll invokes the shader manager singleton, it doesn't exist in the DLL memory space even though it has been called in the main application linking to the DLL. In other words the singleton is not shared by default. There is a good post here about it.

To test this was the case I quickly created a static global variable, within the transform class which could be set outside the DLL module by a quick method call. This basically passed in the application side instance of the Shader manager class, and used that in the call, and it fixed the problem. However Rob the Bloke pointed out the error of my ways by saying

"That's a critical bug and refactor waiting to happen. If you need to share something across DLL boundaries, there is exactly one way to achieve that. Your solution fails the second anyone adds another DLL that links to ngl (i.e. you've just offset the problem for your students to deal with later). s_globalShaderInstance *needs* to be DLL exported."

Making it a "proper" dll

It was now time to bite the bullet and do a major re-factor of the code to make all the classes export from the DLL properly, I've never done this before and the initial attempts were somewhat problematic, however the basic approach is as Rob said

"if you need to access something from a DLL, it needs to be DLL_EXPORTED. End of story"

This code would have to be in the windows version, and depending upon wether we are building the library (DLL) or using the library we would need to either __dllimport or __dllexport our classes.

So we need some pre-compilation glue to make this work, first under windows only then dependent upon if we are building the dll or using the dll. The following code was added to the Types.h class

Once this has been added to the types file we need to add the BUILDING_DLL define to each of the classes. This is shown below.

class NGL_DLLEXPORT Vector
{
...
};

Once this was added it seemed to work for all the classes in the basic demo. However some of the more advance demos still failed to compile. I have several functions in the library which also needed to be exported, this was done as follows

Or so I thought, some of the simple programs that only used the ngl library compiled and ran correctly, however one of the programs which made a call to glUseProgram(0) in the main application failed to compile, due to a linker error, this error said that the method was not found and gave an imp__glew.... style error which means that the actual function call could not be found.

I thought that I had statically bound GLEW into my lib so it would work, however GLEW basically binds method calls to the driver once the glewInit function is called and in this case this is called in the DLL but no my application, to overcome this problem I need to put the glew.c source code into my applications as well the library.

To do this the following is added to the application .pro file (as the glew.c file is shipped with ngl)

This is not an ideal solution but is quite easy to do and saves the hassle of building the glew library using Qt / mingw. I will at some stage do this but for now this quick hack will suffice.And for my next trick
Now that I've sullied my mac by installing windows I think I need to do a bit more windows support, the next stage is going to be building a Visual Studio version of ngl. Once this is done I think I may also add a DirectX back end to it as well however for now I will stick to the mac ;-)

Tuesday, 22 March 2011

This blog post is going to be my first Video blog post, it show how to start from scratch and create a gui program using Qt and NGL, the videos have been posted to youtube as the quality is much better than on blogspot, and you can grab the code using bzr from here

bzr branch http://nccastaff.bmth.ac.uk/jmacey/Code/QtNGL

For my students I've also copied the original movie files onto the public directory here /public/mapublic/jmacey/QtNGL/

Part 1 Setup

This video show how to setup the initial project and directories, it uses the ncca coding standard for project setup

Part 2 Adding source files to the project

This video shows how to add our source files as well as an initial QMainWindow class and a main.cpp file

Part 3 Setting up the .pro project file

This move shows how to setup the .pro file for qmake and we write a simple QApplication within the main.cpp file

Part 4 Creating a GUI using the Designer

In this Video I create the inital part of the GUI using the QDesigner module of QT Creator, I also go into quite a lot of detail about using the QGridLayout and the use of Layouts in general

Part 5 Adding the rest of the GUI components

In this video I add the rest of the UI components and finish off the rest of the GUI.

Part 6 Creating the OpenGL Window

In this video we extend the QGLWidget class to build our own GLWindow which we will put in the NGL classes to draw a teapot / sphere and cube and adjust the properties of the ngl::Transform Class. In this initial part of this video we add the GLWindow to the grid layout. I've added an empty GLWindow.cpp/h file here

Part 7 Adding NGL to the .pro file

Next we add lots of flags to the .pro qmake file to configure the paths used for the ngl library.

Part 8 Adding a Camera

Now we add an ngl::Camera class and a simple ngl::Primitive drawing routine

Part 9 Adding glsl Shaders

Using the ngl::ShaderManager class we load the vertex and fragment shaders and give them a quick test

Part 10 Adding Signals and Slots

We now use the Qt Signals and Slots mechanism to wire up the GUI components to our GLWindow class

Part 11 connecting the rest of the GUI components

Using the signals and slots mechanism I wire up the rest of the components

Part 12 Connecting the combo box

Using signals and slots I connect the QComboBox to select the different drawing modes

Part 13 Lights and Colour

In the final instalment I add an ngl::Light and Material to the program to light and set the initial material values, next I use the QColorDialog to select the colour I wish to use for the object

Monday, 14 March 2011

It is quite typical in a game to need to control an object by keeping a key pressed and for it to respond to these pressed key continuously. The following example demonstrates this in action as shown in the following video

Space Ship class
The following class diagram shows the basic space ship class we are going to move

The constructor of the class is very simple, it loads in the mesh and sets the member variables to default values as shown

The move method, first stores the current x and y values of m_pos, we then increment the values based on the parameter passed in and check agains the extents of the screen. If we are not at the edge of the screen as set in these extents we add the values passed in to the value of the m_pos attribute.
For the rotation the value is just added.

Processing Key presses

Each subclass of the QWidget class has access to the following virtual protected methods

Which ever QtWindow has focus will receive these events and the current key can be queried via the event parameter passed in. In this example the MainWindow class processes the key events as we wish to allow the window to become fullscreen and this method can only be called from a class that inherits from QMainWindow The following code shows how the key press and release methods are implemented

Once any keys required in the MainWindow class are processed the key events are then passed onto the GLWindow class for more processing.GLWindow key processing
To process the keys in the GLWindow class we are going to create a QSet to contain the active key strokes, for each keyPress event (key down) we will add the key to the QSet using the += operator. Every time is released the key will be removed from the QSet using the -= operator. This is shown in the following code

Now we will create an method to process the key values stored in the set and update the ship.

void GLWindow::moveShip()
{
/// first we reset the movement values
float xDirection=0.0;
float yDirection=0.0;
float rotation=0.0;
// now we loop for each of the pressed keys in the the set
// and see which ones have been pressed. If they have been pressed
// we set the movement value to be an incremental value
foreach(Qt::Key key, m_keysPressed)
{
switch (key)
{
case Qt::Key_Left : { xDirection=s_shipUpdate; break;}
case Qt::Key_Right : { xDirection=-s_shipUpdate; break;}
case Qt::Key_Up : { yDirection=s_shipUpdate; break;}
case Qt::Key_Down : { yDirection=-s_shipUpdate; break;}
case Qt::Key_R : { rotation=1.0; break;}
default : break;
}
}
// if the set is non zero size we can update the ship movement
// then tell openGL to re-draw
if(m_keysPressed.size() !=0)
{
m_ship->move(xDirection,yDirection);
m_ship->rotate(rotation);
}
}

First we set the x,y and rotation values to 0 to indicate there is no movement to the ship, next we loop through all the keys in the QSet using the foreach iterator provided by Qt. We then check to see if a key is active and update the direction variable based on the static variable s_shipUpdate. Depending upon the direction we set this to +/-s_shipUpdate.
Finally we check to see if there are any keys active and if so update the ship direction by calling the move function.

Tying it all together
Finally we want to separate the movement and draw commands so we can get a fixed update for drawing. To do this we create two timers one which will trigger the ship move update and one for the re-draw. This is done with the following code, first in the constructor we start the timer.

m_updateShipTimer=startTimer(2);
m_redrawTimer=startTimer(10);

Next in the timerEvent in GLWindow we process and dispatch if we get the correct timer event as follows

void GLWindow::timerEvent(
QTimerEvent *_event
)
{
// the usual process is to check the event timerID and compare it to
// any timers we have started with startTimer
if (_event->timerId() == m_updateShipTimer)
{
moveShip();
}
if (_event->timerId() == m_redrawTimer)
{
updateGL();
}
}

The full code for this demo can be downloaded from here using the command bzr branch http://nccastaff.bmth.ac.uk/jmacey/Code/GameKeyControl