Creating an OpenGL Overlay

I want to create an app that displays an overlay on top of a game using OpenGL. There some programs in the Windows-world which can do this (Xfire, Teamspeak Overlay). Afaik they are doing this by hooking some OpenGL (or Direct3D) calls and replacing them with their own code.
IÂ´ve searched a bit around, and the easiest way to do this seems to be the Application Enhancer: http://unsanity.com/haxies/ape
ItÂ´s quite easy to use, IÂ´ve already managed to hook gl_Begin and draw my own stuff in the window of a running game, but hooking gl_begin isnÂ´t very efficent, since it gets called once for each object per frame.
So my question is: Do you know an OpenGL function which gets called (in most games) once per frame? Or is there even a better method than doing it like this?

But the only games (beside my OpenGL projects) are the newer Feral games like Tomb Raider or Race Driver 3. In all other games IÂ´ve tried IÂ´m only getting the white rect during startup, or when the games are showing the intro video.
IÂ´ve tried UT2004, CoD 1&4, Quake 3/ioQuake3, Urban Terror, Wolfenstein:ET, but none of these games work correctly with my overlay, IÂ´m only getting it during startup with some of them...
IÂ´ve also hooked glFlush, glFinish, glSwapAPPLE and aglSwapBuffers, but it doesnÂ´t change anything...
What the hell are all these games calling when they reach the end of their drawing function?! Do you know any functions beside these IÂ´ve already hooked? Or is there maybe a mistake in my overlay drawing code?

Is your code still being called in newer games, and the rendering doesn't appear? Or is your code not being called at all.

There are a few hundred other things in the context state that could cause your rendering to be broken. For example, if the game left a vertex shader enabled when they swap. You're not checking for that and disabling it. Your rendering is going to use whatever state vector they left set up.

Oh, thanks for the hint.
CGLFlushDrawable seems to get called in all games, but the rendering doesnÂ´t work.
I thought I was hooking the wrong functions, but CGLFlushDrawable seems to be okay.
So the problem is within my rendering code...
Thanks for your help

I got a new problem...
Since the APE seems to be a bit buggy (it made my iMac hang up when I tried to boot, only removing APE in the single user mode helped) I decided to use another method: Compiling my code as Dynamic Library and loading it with the dyld environment variable "DYLD_INSERT_LIBRARIES" when an application starts.
This works for functions which get called directly in the games code, e.g. glBegin or glEnable, for some games [NSOpenGLContext flushBuffer] works, too.
But IÂ´m not able to override "CGLFlushDrawable()" using this method. I think it doesnÂ´t work, because no game seems to call "CGLFlushDrawable" directly. "CGLFlushDrawable" seems to get called by NSOpenGL or AGL or GLUT. Just adding the function "CGLFlushDrawable" to my code works for Application Enhancer, but not when I load the code using dyld.
Do you know whether there is a way to override CGLFlushDrawable using the dyld-method? Or is there maybe a better method than using dyld? I found this code, but it doesnÂ´t work for x86-Macs and itÂ´s quite difficult. I could also try APE again, but I donÂ´t want that the users of my app have to download APE to be able to use my app, so dyld seems to be the better way...

Can you clarify your goal here? You want to do something like Xfire, where you display a chat dialog or some other UI composited on top of an arbitrary OpenGL application?

I think this is difficult to do in a robust way.

It sounds like you're interposing specific functions (CGLFlushDrawable etc) and then inserting draw commands into the app's context. This approach is doomed to failure, because if you use the app's context, you need to save/restore their state around your drawing. And there's no future-proof way for you to save all of the state correctly-- even if you carefully push/pop/get/set every state variable that exists today, a future extension may add a non push-poppable bit that will break your rendering.

A more robust way would be use your own context for the drawing. You can set all the state there without worrying about what the app is doing. The problem with that approach is that you need to attach your context to the app's current drawable. You can do this if you interpose at the NSGL or AGL level and the app runs in a window, but you can't do it at the CGL level or if the app is fullscreen, because there's no public API to get the fullscreen drawable a context is attached to.

arekkusu Wrote:Can you clarify your goal here? You want to do something like Xfire, where you display a chat dialog or some other UI composited on top of an arbitrary OpenGL application?

Yep, thats what I want to do

arekkusu Wrote:It sounds like you're interposing specific functions (CGLFlushDrawable etc) and then inserting draw commands into the app's context. This approach is doomed to failure, because if you use the app's context, you need to save/restore their state around your drawing. And there's no future-proof way for you to save all of the state correctly-- even if you carefully push/pop/get/set every state variable that exists today, a future extension may add a non push-poppable bit that will break your rendering.

A more robust way would be use your own context for the drawing. You can set all the state there without worrying about what the app is doing. The problem with that approach is that you need to attach your context to the app's current drawable. You can do this if you interpose at the NSGL or AGL level and the app runs in a window, but you can't do it at the CGL level or if the app is fullscreen, because there's no public API to get the fullscreen drawable a context is attached to.

Using this method (overriding CGLFlushDrawable) worked quite good for me when I did it in App Enhancer. It worked for all Quake3/ioQuake3-based games, many old OpenGL-Games and for example Doom3, Quake 4, Call of Duty 4 and many more. I only had problems with UT2004-based games (they simply donÂ´t show my overlay), but support for a few games is still better than no overlay at all.
That isnÂ´t my main problem, currently my problem is that I canÂ´t override "CGLFlushDrawable" with dyld as described above...
If this is simply not possible, then IÂ´ll try to override the functions in AGL/NSOpenGL which flush/swap the buffer(s). The problem is, that I donÂ´t know every function which can do that.
Are there some more beside glFinish, glFlush, glSwapAPPLE, aglSwapBuffers, glutSwapBuffers and [NSOpenGLContext flushBuffer]?