Good catch! You're right, I missed that. There is indeed a tiny window of opportunity for the queue to be transferred in the link thread, then set in main thread, then re-cleared in the link thread. Gonna have to add a lock... I should know better.

Code:

keyDown[i] = keyDownQueue[i]; // pretend keyDownQueue[i] == NO
// main thread catches new keyDown here so now keyDownQueue[i] is YES
if (keyDown[i]) //not relevant but taking time
{
key[i] = YES; //not relevant but taking time
}
keyDownQueue[i] = NO; // whoops! We just missed that the main thread said YES!

That said, I should note that I've been using this for threaded updates for years and have never once missed any input that I noticed. I guess that tiny window gets passed so quickly that the odds of it happening are just too low to have seen it... yet. Plus, missed input isn't a crash-inducing thread conflict, so I guess if it had happened, I may have just been confused about the game's response, and hitting the key again would've reset it back to sanity.

Threading is fun!

[edit] There, I added some locking to make sure the two threads don't pull any switcheroos on each other. I hope that looks a little more reasonable

[edit2] Whoops, never mind... yeah, I do use locks in my threaded code. I use the same queues in my non-threaded code, but obviously don't need the locks there, so I forgot to add them when I whipped this example together.

AnotherJake Wrote:Speaking of "smoothness", you may notice that the fixed rate timing isn't quite as glass smooth as just using the delta time from display link alone.

Just to be sure, using the delta time from the display link alone would be like [spaceship moveForTime: deltaTime], right?

I haven't thought about sub-update interpolations before. At least Bullet has it, though I still have to worry about the non-Bullet-using parts of my game. I suppose it wouldn't matter though if I'm never changing the logic update rate; sub-update interpolations seem more important for when I want to alter the physics rate in-game for effects like slow-motion.

What unit is the frame delta time calculated from the display link in anyway?

I've noticed in your code you get frameDeltaTime from the display link, but you also have currTime and prevFrameTime to demonstrate how to get the frame delta time without it. I could just remove references to those variables, as well as the method currentTime, if I use nothing but the display link, right?

When the view reshapes, where would I put the code that changes the OpenGL view? In your code you set the projection matrix every time you draw; would it be better to set the viewport after the line [[self openGLContext] update] but before you unlock the the context in the reshape method?

As for the locks around the key input code, say I've separated your code into a view for rendering and timing, a controller for keyboard input, and a model to store game state. The view tells the game to update and render itself, the controller passes keyboard events to the game. In this case, would I be putting the lock around the line that that passes the key event to the game?

Coyote Wrote:Just to be sure, using the delta time from the display link alone would be like [spaceship moveForTime: deltaTime], right?

Yes. Well, you mean if *not* using the fixed rate version I assume. This stuff is so subtle The snag with deltaTime is that there is only one deltaTime when not using the fixed rate timing, and then there are *two* deltaTime's when you *are* using fixed rate timing. That's why I try to differentiate it as frameDeltaTime instead of just deltaTime. deltaTime, or dt for short, almost universally implies that you're using it for your animation calcs, like: in your spaceship method you do something like locationX += deltaTime * velocity;

Coyote Wrote:I haven't thought about sub-update interpolations before. At least Bullet has it, though I still have to worry about the non-Bullet-using parts of my game. I suppose it wouldn't matter though if I'm never changing the logic update rate; sub-update interpolations seem more important for when I want to alter the physics rate in-game for effects like slow-motion.

Well, the sub-update interpolations, or "tick interpolations" as I call them sometimes, are really more for getting rid of temporal aliasing. If you look at the Pong example really closely while it's running, in fixed-rate mode, if it's not the same tick rate (update rate) as the display refresh, you'll see what look like tiny little steps in the motion. It won't be as glass-smooth as if you use variable rate deltaTime (i.e. not using the fixed-rate technique). The tick interpolations are to smooth that motion back out to how it was when using variable rate, but again, it takes more work. I've used tick interpolations myself, but haven't made it standard practice yet because I don't have a standard system I'm happy with to blend states. As I mentioned, you probably don't even need to use tick-interpolation for most games because after all the action, the users probably won't notice the tiny amount of temporal aliasing. Heck, even as a developer, if you don't recognize the phenomenon, you'd probably not care much yourself!

When you'll really want to start using tick interpolations however, is if you're doing networked multiplayer, where you'll likely have no choice since snapshots from the server might only be 20 Hz. But that's a whole 'nuther can of beans!

Coyote Wrote:What unit is the frame delta time calculated from the display link in anyway?

double precision seconds.

Coyote Wrote:I've noticed in your code you get frameDeltaTime from the display link, but you also have currTime and prevFrameTime to demonstrate how to get the frame delta time without it. I could just remove references to those variables, as well as the method currentTime, if I use nothing but the display link, right?

Yes, you don't have to use mach time (i.e. you don't need the currentTime method) if you're basing your time off the display link... I *think*... I say that because I don't know if there are variations of the time base coming from display link since I haven't used it enough in practice; although now that I finally see it might be worth it, I'm making an effort to shift some of my code base over to it and see how things go from here.

Coyote Wrote:When the view reshapes, where would I put the code that changes the OpenGL view? In your code you set the projection matrix every time you draw; would it be better to set the viewport after the line [[self openGLContext] update] but before you unlock the the context in the reshape method?

Yeah, that sounds reasonable if that's what you want to do. I just set the matrix every time because it's only done once per frame, which in my experience makes it a pretty negligible operation. Plus, sometimes there are effects I use where I change the projection. I like keeping as much of my GL code co-located too, so looking at a file and not seeing the projection there too kinda bugs me sometimes -- purely subjective though.

Coyote Wrote:As for the locks around the key input code, say I've separated your code into a view for rendering and timing, a controller for keyboard input, and a model to store game state. The view tells the game to update and render itself, the controller passes keyboard events to the game. In this case, would I be putting the lock around the line that that passes the key event to the game?

That I don't know for sure. Depends on how you implement everything. If you send events directly to the game, you need to remember that the game is updating on a separate thread. That means the game might be going along thinking UP_ARROW is being pressed at the beginning of the update and it magically stops being pressed by the end of the same update because the main thread just sent a new input event somewhere in the meantime. That's why I prefer to queue the input as I demonstrated -- it gives the entire update the same picture of what the input looks like, and any new changes in input during that update will be picked up when the next update starts, so nothing gets missed.

As for worrying about the locks, if you don't have much experience with threaded programming, luckily, thread conflicts with input don't cause crashes. That said, if you *are* relatively new to threaded programming, taking out the locks in my example and then reading imron's post should give you a great case-study in what can happen when "things go wrong ".

Which brings up another point: If you're having problems with mystery crashes while using display link, you might consider making an alternate path using a standard NSTimer. Making the alternate path is not hard and you can use the same timing algorithm; just that the timer isn't as high priority as the display link, so you're more likely to drop a frame here and there (although even with the occasional dropped frame, one of the beauties of fixed rate updates is that they help cover up the visual effect of the dropped frame). That would keep everything on the main thread which might make debugging easier. It won't be perfectly as smooth as the display link can be, but it ain't bad either -- many of us have been doing it that way for years. Personally, I will keep an NSTimer path in my code alongside the display link for exactly this debugging purpose.

// set to 0 to use standard NSTimer instead, perhaps for debugging
#define USE_DISPLAY_LINK 1

// set to 1 for fixed rate, 0 for variable delta (variable delta looks smooth, but is inaccurate for simulation)
#define USE_FIXED_RATE_TIMING 1

// arbitrary Hertz -- you can pick whatever rate you want. I like 110 because it seems to be the
// smoothest compromise across different display refresh rates, but that is completely subjective
#define TICK_RATE 110.0

// we stop updating below this frame rate, which effectively pauses the game until drawing catches up,
// so not a good idea to go too high with this constant
#define MINIMUM_FRAME_RATE 4.0

// calculate the time base for this platform only on the first time through
if (timebase == 0)
{
mach_timebase_info_data_t timebaseInfo;
mach_timebase_info(&timebaseInfo);
timebase = timebaseInfo.numer / timebaseInfo.denom;
}

// set up our fixed delta time (which we maintain as an instance variable in case you want to change
// it on the fly somewhere down the road, for whatever reason -- you could just as well use a constant
// or define if you prefer instead)
dt = 1.0 / TICK_RATE;

@coyote, if you're relatively new to concurrency, you might also want to check out Herb Sutter's articles on the topic. See the list of articles here (click for link). Although the articles have an emphasis on C++, a lot of the core principles and theories can be applied to any language.

Also, another good way to actually experience the problems mentioned above when there are no locks around the input, is to add in a forced context switch at the relevant position. For example, if you take the locks out of AnotherJake's code and then in getInput add something like:

Code:

if ( i == LEFT_ARROW )
{
[NSThread sleepForTimeInterval:0.005];
}

just before the line:

Code:

keyUpQueue[i] = NO;

then you greatly increase the likelihood of seeing the problem I mentioned in my post above, and you'll regularly get missed keyUps for the LEFT_ARROW, but not any others (haha, that's how I caught the missing [[NSLock alloc] init]). As AnotherJake mentioned, it's not going to cause your application to go bang, and it's probably never going to happen to most of your users, but randomly missing input/sticky keys is one of those problems that can be a nightmare to track down.

imron Wrote:Also, now that you've added locking, doesn't it make having the extra keyUp/Down arrays redundant? e.g. in getInput why not just do:
...
Having the extra array doesn't appear to really do anything.

That is a little subtle, I admit (unless I'm totally missing your point, which is entirely possible at this hour). The difference is that keyUp/keyDown aren't "sticky"; they get cleared each and every update, whereas key stays down until keyUp/keyDown change -- key can remain persistent across frames. You *could* do just

Code:

if (!ballMoving && key[SPACE])

and it *would* work just fine in this particular example. There are times when you only want to catch a key down as one toggle of an option, like "pause". If you don't use keyDown and use key instead it'd pause/unpause every update!

Quote:There are times when you only want to catch a key down as one toggle of an option, like "pause". If you don't use keyDown and use key instead it'd pause/unpause every update!

Ah, fair enough. I guess it then comes down to how many toggle keys you have, and does it warrant having two extra arrays (keyDown/keyUp) instead of just a single variable for the ones that need it,
or perhaps a middle ground approach of having a single keyPressed array cleared each frame rather than both a keyDown and keyUp.

Now that you've mentioned it though, having both arrays does provide the most flexibility in allowing to check for either keyDown or keyUp events for a given update frame.

Yeah, there's definitely a little redundancy there. You might not ever need keyUp, but I've found that having the "toggle" array (keyDown) is very convenient, in addition to having key. Obviously, one can rename them to anything they like, like you said, maybe keyPressed instead.

A trick I forgot to mention is that you can "consume" key presses. To consume a key, just set it to NO after you use it. That way, if you want to make sure nothing else responds to that key later on in that update, you don't have to find other objects that may respond to it and special case them. Like for instance, let's say you have a chat console up temporarily. You probably don't want the spaceship to be firing while you're busy razzing your opponent, so if the chat console is up, in your code you can just set all the keys to NO after the chat console code. That way you don't have to special case everything like if (key[SPACE && !chatConsoleActive]) spaceShipFire().

Another case where you might want to "consume" a key is during development, when you're testing things out. It might be convenient to use a key for a test feature, then consume it so it can't interfere elsewhere in the code. I've used that a lot!

OR, let's say you toggle the 'c' key to enable camera control instead of player control. During camera control you use the WASD keys and consume them right afterwards so they can't be used to control the player later on in the update at the same time. Key "consumption" can be handy!

Thanks for posting some DisplayLink code - it motivated to start using it on my next project... :-)

I do have a conceptual question: Is all of the rendering and updating occurring on the corevideo displaylink callback thread rather than the main app thread?

I have tried calling [self setNeedsDisplay:true]; directly from the displaylink callback -> this results in drawRect being called by the main app thread without needing any context locking. I then do all my drawing and rendering from there on the main thread.

Is this a valid approach?

I had some strange lockups when trying to render using the displaylink thread and context locking...

OptimisticMonkey Wrote:Thanks for posting some DisplayLink code - it motivated to start using it on my next project... :-)

I do have a conceptual question: Is all of the rendering and updating occurring on the corevideo displaylink callback thread rather than the main app thread?

Yes.

OptimisticMonkey Wrote:I have tried calling [self setNeedsDisplay:true]; directly from the displaylink callback -> this results in drawRect being called by the main app thread without needing any context locking. I then do all my drawing and rendering from there on the main thread.

Is this a valid approach?

I don't think so. I mean, if you're just throwing it back to the main thread to render when it gets around to it, then you're kind of defeating the purpose of using the display link for high priority timing aren't you? Plus, I'd be wary of it because...

OptimisticMonkey Wrote:I had some strange lockups when trying to render using the displaylink thread and context locking...

That brings up a very important point which hasn't been discussed yet: Don't assume Cocoa is thread-safe! Don't touch any Cocoa stuff on the main thread from another thread without knowing if that action is safe!

If you touch Cocoa from another thread, like the display link thread, you might very well get a lock-up. There are some things that are thread safe, but generally anything that has to do with the UI or stuff that resides on the main thread should be avoided like the plague (or at least carefully investigated for safety first). Check out what Apple says about it.

[adding] My experience with this has shown that with few exceptions, if I access Cocoa from a secondary thread, like I would from the main thread, without thinking about it, whacky crashes ensue. I've found that developing in the secondary thread means taking things one step at a time with any Cocoa stuff so that if a crash occurs I can back up just a bit to where it wasn't crashing. This is where multi-threaded programming can be really tricky. Cocoa crashing because of a thread conflict can be *really* hard to track down (if not impossible), so it's best to move slowly and methodically. It takes some practice to know what you can and cannot do. This is also why I prefer to maintain a non-threaded path alongside the threaded path if I can, like in this case having the option to go back to using the regular 1kHz NSTimer for debugging.

I still think that using setNeedsDisplay from the callback might be a good idea.

Here is my thinking:

setNeedsDisplay flags the view as being dirty.

I believe that Quartz automatically calls drawRect on all "dirty" views every frame, once per frame, no matter what.

Now the problem with using an NSTimer to flag the view as dirty is unreliable, because it doesn't fire consistently with the refresh rate. But using the displaylink callback to flag it as dirty should result in perfect syncing with the vertical refresh.

I think using the main app thread for rendering and updating really simplifies everything and makes debugging easier. There is no locking, and the main thread is used for rendering update.

What do you think?

Thanks again for posting your displaylink code - like I said it really motivated me to dig in and learn it...

OptimisticMonkey Wrote:I think using the main app thread for rendering and updating really simplifies everything and makes debugging easier. There is no locking, and the main thread is used for rendering update.

What do you think?

I guess if it works, why not? [shrugs]

Like I mentioned above, my main two concerns with the approach:
1) Is the performance actually better than a standard NSTimer when doing it this way? I don't know. I haven't looked into it myself yet.
2) Talking to Cocoa on the main thread for anything is a *scary* proposition. Even just setting needsDisplay makes me wary. But hey, maybe it's okay, I don't know...

OptimisticMonkey Wrote:Thanks again for posting your displaylink code - like I said it really motivated me to dig in and learn it...

Right on. Always glad to help. I don't know if my way of doing it here is whacky though, so use it with some caution. Your idea of going back to the main thread with it might be best, I really don't know. I haven't used display link much myself before this, except for a few tests in the past.

All it does so far is draw a yellow background. I also have some untested fullscreen code. By the way, when I switch to fullscreen mode, is the view the dimensions of my screen or the last dimensions of the window?

I ended up basing my timing code more on the Sacred Software guide. I've also decided to not use a separate object for keyboard input. What I'm probably going to do instead is keep an array of the currently pressed keys where key codes are added and removed in the key event methods. The key code array would get passed to the game model with its update message in the getFrameForTime method (called drawFrame in my code). When passing the array along, would it be better to lock then create a copy of the array first, then pass the array copy to the game model?

Given the number of locks in this code, do you suppose introducing Grand Central Dispatch would be worth it or possible? GCD is apparently supposed to eliminate for locks, but I don't have enough experience to tell if that'd apply to this instance (really what I'm doing is copying your lock code and hoping it works. )

I also had another question about how you handled the reshape function. I notice it is here that you update the view's NSOpenGLContext (in between a lock and after setting it to be the current context). I also notice that you haven't overridden the view's update method.

According to the docs, this is where the view's NSOpenGLContext is updated, and the doc says that you may "add locks for multithreaded access to multiple contexts."

Since update is called when the view is moved and resized, I wonder if it's possible to move your context management there in order to simplify the reshape function (which is where I'd really prefer putting my viewport and display frustum in).

Yeah, you can say that again. I think this timing discussion and related subjects are some of the more subtle subjects to tackle when it comes to making games. I've seen a lot of different approaches over the years, read a lot of articles and books, and seen a fair amount of flamewars over it. Others may disagree, but I think it's pretty hard to get it just right.

Coyote Wrote:By the way, when I switch to fullscreen mode, is the view the dimensions of my screen or the last dimensions of the window?

You'll be setting your glViewport to the screen size. Switching into full screen can be a new subject all unto itself if you ask me!

Coyote Wrote:When passing the array along, would it be better to lock then create a copy of the array first, then pass the array copy to the game model?

I would. That way you don't have to lock it if your update is in another thread from where the input is received. That's why I use the "queues" technique that I described, which sounds similar to your plan. The idea is to give a "snapshot" of the current user input for the frame. If you ever do multiplayer network development, that's what you'll be doing anyway when you package up input to send to the server (assuming a client-server model of course).

Coyote Wrote:Given the number of locks in this code, do you suppose introducing Grand Central Dispatch would be worth it or possible? GCD is apparently supposed to eliminate for locks, but I don't have enough experience to tell if that'd apply to this instance (really what I'm doing is copying your lock code and hoping it works. )

You really need to study up a little more on concurrency. "copying lock code" sounds like a recipe for disaster. Here's a quick intro to how it works in this Pong example:

We have data that we don't want touched by two different CPUs at the same time. In this case it is that we don't want the display thread touching the keyDownQueue or keyUpQueue at the same time as the main thread, and vice-versa, until both are done with their operations since the result in this case may be a "sticky" key that we didn't expect. For the sake of discussion, let's just talk about keyDownQueue. So just before any code is going to touch keyDownQueue we try to "acquire a lock" first. Let's pretend the main thread already acquired the lock and is currently adding input to the keyDownQueue, and the display link thread is about to touch keyDownQueue to copy from it -- that would be a conflict which we don't want to happen. The display link thread calls [inputLock lock] to try to acquire the lock for itself but since inputLock has already been acquired by the main thread, the display link thread waits until the lock is available (it "blocks" the thread until the lock is relinquished). Once the main thread is done messing with keyDownQueue it calls [inputLock unlock] to release the lock, and the display link thread sees this and successfully acquires the lock for itself. If by chance (pretend here) that the main thread comes back to mess with keyDownQueue again, it won't be able to acquire the lock until the display link thread is done with keyDownQueue and calls [inputLock unlock].

So any time you are trying to protect a piece of data from being touched by code being run on two different threads, you bracket that code with lock/unlock to mediate access to it. This obviously isn't ideal since if a thread encounters a lock that's already been acquired it has to wait. The waiting does cost a bit of performance, obviously, so you want to use locks as sparingly as possible.

Coyote Wrote:I also had another question about how you handled the reshape function. ...
Since update is called when the view is moved and resized, I wonder if it's possible to move your context management there in order to simplify the reshape function (which is where I'd really prefer putting my viewport and display frustum in).

Go ahead and give it a try! I put it in reshape because that's where it's been proven to work best for me under the varying conditions I've needed it for. YMMV