Post navigation

Today we journey as pythonauts into the world of particle systems and fast 3D graphics, manipulating thousands upon thousands of little dots in mere milliseconds! We accomplish this with two Python modules, PyOpenCL and PyOpenGL. This is a port of my C++ OpenCL tutorial, but with much less code and all the splendors of numpy.

Let’s get started!

You can go ahead and run python main.py to see the code in action. The way our particle system works is that we have some collection of particles, really just 3D points in space, each with it’s own lifetime. Every time we loop we want to update the positions of our particles based on some set of rules (e.g. gravity, initial velocity) and decrease their lifetime a little. When the lifetime of a particle reaches 0 we set it back to its original position and reset its lifetime. So we want to structure our code so that we initialize the positions of the particles first, then every frame we update and render them.

main.py is responsible for setting up the OpenGL environment using PyOpenGL and GLUT, providing mouse and keyboard interaction as well as a run loop for our program. So now we have some 3D space we can look around by clicking and dragging (left and right mouse buttons) or hit ‘q’ to quit and ‘t’ to print out timing of the update function.

The Setup

Once we have our environment we need to setup some particles, for this look in initialize.py. The first function you will see is function_np which creates several numpy arrays and sets their values using numpy slice operators. If that’s a little confusing, it may be easier to follow the fountain_loopy function which does the same thing but with a (slower) for loop. Essentially we are randomly placing the particles on a flat donut in the x, y plane. They each start with an initial velocity in the same direction as their position and a random lifetime. We also make an RGBA color array so that each particle can have its own color.

Notice the fountain function at the bottom, which just calls the fountain_np function and creates Vertex Buffer Objects (VBO) out of the numpy arrays. Presumably when you go on to make your own OpenCL/OpenGL code you will start with your own numpy arrays and this is a key point for setting up your memory on the GPU. If you want OpenCL to be able to act on OpenGL memory (without making copies) you will need to prepare VBOs (or render buffer objects with clImage which should be the subject of a later tutorial) before you go to OpenCL. Luckily its dead simple with PyOpenGL

Here num and dt are defined at the top of main.py, which are user parameters for how many particles to make and essentially how much simulation time passes each frame. Now we get to the OpenCL, we’ve made an instance of a Part2 class. Then we call loadData with the vbos and velocity array.

Interfacing with OpenCL

Let’s take a look at what the constructor and loadData are doing in part2.py

First we get a list of available OpenCL platforms on the machine and then import a handy function provided by PyOpenCL which abstracts the messy business[pdf] of setting up the right properties for sharing a GL context. Due to the eccentric whims of Apple (hey, they pretty much came up with OpenCL) the way you create the context on Mac OS X is slightly different, hence the if statement checking for “darwin”. After that its back to business as usual by creating a CommandQueue from the context. If you are using PyOpenCL .92 or beta2011 you will need to replace the contents of the clinit function with this code.

So loadProgram is the same as in Part 1 where we simply read in the file to instantiate and build a program object. So lets skip that and get straight into loading the data:

The key here is on line 33 in creating OpenCL buffers from the vbo objects using the cl.GLBuffer class. Notice how we do not pass in the vbo object itself, but the actual integer value of the buffer as it is represented by OpenGL. We create normal OpenCL buffers as we did in Part 1 simply by passing in numpy arrays (♥). The other significant change is defining the gl_objects list which we will use in execute!

The first thing to point out is that we are acquiring the gl_objects before we pass them in as arguments to the kernel. This makes sure that OpenGL is not using the buffers for anything and allows us to read from and write to them. We are setting our global workgroup size to the length of our arrays, essentially saying that each thread global workitem will be one element of our original array.
We also introduce sub intervals, allowing us to perform a variable number of updates per frame. This means one could make dt smaller, generally making the simulation more accurate but not slow down the desired motion of the particles. For example if you decrease dt from .01 to .001 you will be making the particles move 10 times slower every iteration, so to keep the visual speed the same you would do 10 sub iterations.

The Kernel

So after all that setup we are ready to run our kernel, found in part2.cl.

So there you have it, the essentials of interoperating between OpenCL and OpenGL in Python! I didn’t cover what I did in my utility files but those are subjects of later posts. You should be able to poke around to see whats going on, and as far as rendering I’m using very basic GL calls for VBOs which there are othertutorialsfor.

I’d like to shout out to Keith Brafford for helping test and refactor this code on Windows as well as the PyOpenCL patch I worked on to get GL interop working on the Mac. Of course this tutorial wouldn’t be possible without the valiant efforts of Andreas Klöckner!

Post navigation

21 thoughts on “Adventures in PyOpenCL: Part 2, Particles with PyOpenGL”

I tried to execute main.py but i get this result:from pyopencl.tools import get_gl_sharing_context_properties
ImportError: cannot import name get_gl_sharing_context_properties
I’ve searched all over the Internet but couldn’t find anything?

Hello.
I followed the steps in the tutorial but failed to run the program.
I get this error:
File “main.py”, line 135, in
p2 = window()
File “main.py”, line 56, in __init__
(pos_vbo, col_vbo, vel) = initialize.fountain(num)
File “/home/radu/Desktop/Python/adventures_in_opencl/python/part2/initialize.py”, line 84, in fountain
pos_vbo.bind()
File “vbo.pyx”, line 227, in OpenGL_accelerate.vbo.VBO.bind (src/vbo.c:2590)
File “vbo.pyx”, line 181, in OpenGL_accelerate.vbo.VBO.create_buffers (src/vbo.c:1872)
File “latebind.pyx”, line 32, in OpenGL_accelerate.latebind.LateBind.__call__ (src/latebind.c:559)
File “wrapper.pyx”, line 308, in OpenGL_accelerate.wrapper.Wrapper.__call__ (src/wrapper.c:5059)
File “/usr/lib/pymodules/python2.7/OpenGL/platform/baseplatform.py”, line 340, in __call__
self.__name__, self.__name__,
OpenGL.error.NullFunctionError: Attempt to call an undefined function glGenBuffersARB, check for bool(glGenBuffersARB) before calling

@Jake, @Aleksander I don’t have Ubuntu available at the moment to test with you, but I just updated to the latest source and everything still works on my mac. Those may be problems with PyOpenGL, I’m trying to remember on a school Ubuntu 11.04 machine I had to build PyOpenGL from source. That might address your genbuffers stuff @Radu unless your graphic cards drivers aren’t up to date.

I just bought a new laptop, with an NVIDIA GT555M card, running Windows 7 x64.
I initially had issues running the Python part2 particle demo, with an error similar to some above.
The actual error I got was:
pyopencl.LogicError: Context failed: invalid gl sharegroup reference khr

I realized after a bit of worrying that the problem was in the Optimus device that the newer laptops have, that is the integrated powersaving graphics card, if I understand correctly.
I was however able to fix that by using the NVIDIA control panel and changing the Global settings to always use the NVIDIA GT555M card, instead of the integrated Optimus card. Then I was able to run the demo without the error message.

“I’ve installed pyopencl 2011.1 and when i run main.py I get this error:
AttributeError: type object ‘context_properties’ has no attribute ‘GL_CONTEXT_KHR’”

But I finally did it. It indicates that either your device doesn’t support OpenCL/OpenGL interoperability, or that PyOpenCL was compiled without it. To get it to work, I had to download cl_ext.h from the Khronos website and put it in /usr/include/CL/, then alter “siteconf.py” which was created by “configure.py” when building PyOpenCL and set CL_ENABLE_GL to True.

Since I also spent the day installing various versions of NVidia drivers, it’s possible something else also helped, but I’m pretty sure that’s what fixed it.

Hi,
Thanks for your interesting tutorial.
However, I was wondering how you would do the following thing:
at the moment, you have a single kernel doing all the math, but for some other problems, you can not do that.
Say for example that at a given time you want to sum the speed of all particles (silly example, but you get the idea) and stop on a condition for that value.

As you mention it in your code, we sometime need to keep the vectors in GPU memory, do some other computation and go back to the initial kernel.
How would you do that ?
Waiting for the next example :)

I also had the ImportError: cannot import name get_gl_sharing_context_properties first, so I built pyopencl-2011.2 from source.
Then I had the ‘GL_CONTEXT_KHR’ exception, which I tried to solve by following the advice above, but after some trial-and-error this worked for me:
./configure.py –cl-enable-gl

i am on Kubuntu 11.10 (same as Ubuntu), using the most recent nvidia driver.