I've been working on application development with a lot of "retained" GUI systems (below more about what I mean by that) like MFC, QT, Forms, SWING and several web-GUI frameworks some years ago. I always found the concepts of most GUI systems overly complicated and clumsy. The amount of callback events, listeners, data copies, something to string to something - conversions (and so on) were always a source of mistakes and headaches compared to other parts in the application. (Even with "proper" use of Data Bindings/Models).

Now I am writing computer games :). I worked with one GUI so far: Miyagi (not well-known, but basically the same Idea as all the other systems.)

It was horrible.

For real-time rendering environments like Games, I get the feeling that "retained" GUI systems are even more obsolete. User interfaces usually don't need to be auto-layouted or have resizable windows on-the-fly. Instead, they need to interact very efficiently with always-changing data (like 3d-positions of models in the world)

A couple of years ago, I stumbled upon "IMGUI" which is basically like an Immediate Graphics mode, but for user interfaces. I didn't give too much attention, since I was still in application development and the IMGUI scene itself seemed to be not really broad nor successfull. Still the approach they take seem to be so utterly sexy and elegant, that it made me want to write something for the next project using this way of UI (I failed to convince anyone at work :(...)

let me summarize what I mean by "retained" and "immediate":

Retained GUI:
In a separate initialization phase, you create "GUI controls" like Labels, Buttons, TextBoxes etc. and use some descriptive (or programmatical) way of placing them on screen - all before anything is rendered. Controls hold most of their own state in memory like X,Y location, size, borders, child controls, label text, images and so on. You can add callbacks and listeners to get informed of events and to update data in the GUI control.

Immediate GUI:
The GUI library consists of one-shot "RenderButton", "RenderLabel", "RenderTextBox"... functions (edit: don't get confused by the Render prefix. These functions also do the logic behind the controls like polling user input, inserting characters, handle character-repeat-speed when user holds down a key and so on...) that you can call to "immediately" render a control (doesn't have to be immediately written to the GPU. Usually its remembered for the current frame and sorted into appropiate batches later). The library does not hold any "state" for these. If you want to hide a button... just don't call the RenderButton function. All RenderXXX functions that have user interaction like a buttons or checkbox have return values that indicate whether e.g. the user clicked into the button. So your "RenderGUI" function looks like a big if/else function where you call or not call your RenderXXX functions depending on your game state and all the data update logic (when a button is pressed) is intermangled into the flow. All data storage is "outside" the gui and passed on-demand to the Render functions. (Of course, you would split up the big functions into several ones or use some class abstractions for grouping parts of the gui. We don't write code like in 1980 anymore, do we? ;))

Now I found that Unity3D actually uses the very same basic approach to their built-in GUI systems. There are probably a couple of GUI's with this approach out there as well?

Still.. when looking around, there seem to be a strong bias towards retained GUI systems? At least I haven't found this approach except in Unity3D and the original IMGUI community seems to be rather .... .. quiet.

So anyone worked with both ideas and have some strong opinion?

Edit: I am most interested in opinions that stem from real-world experience. I think there is a lot of heated discussions in the IMGUI-forum about any "theoretical weakness" of the immediate GUI approach, but I always find it more enlightening to know about real-world weaknesses.

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.
If this question can be reworded to fit the rules in the help center, please edit the question.

7 Answers
7

Nay. I've done paid gamedev work on an awful 'retained mode' GUI and on an awful 'immediate mode' GUI and although both made me want to tear my eyes out, the retained mode approach is still clearly the better one.

The downsides of immediate mode are many:

they don't make it easy for artists and designers to configure the layout;

they make you mix logic with presentation;

they make it harder to have a single consistent input model;

they discourage complex controls of dynamic data (eg. list views);

layering becomes very awkward (eg. if I call RenderComboBox, the drop-down box can't possibly render yet because it needs to go above the rest of the window - same for tool-tips)l

performance can be poor because calling all these functions to query values that may or may not have changed tends to create many small objects, stressing the memory manager;

...and I can't even imagine how painful it would be to allow someone to drag bits the GUI around and re-order them. You'd have to maintain a data structure telling you where to render everything - which is much like rewriting a retained mode system yourself.

Immediate mode GUIs are tempting for lone programmers who want a quick HUD system, and for that purpose they are great. For anything else... just say no.

@NicolBolas: ok, "that ugly" is a very vague statement and of course, how ugly it really gets depends on the library as well as on its user. Still.. I could not find any GUI system yet with the same elegance and beauty (sorry - also vague) that I found in frameworks for, say.. AI, serialization, navigation or similar complex fields. Any recommendations? (BTW: If I would have to rate "elegance", I'd go for QT. It's still a big source of grief.. but at least well thought-through grief :-D).
–
ImiFeb 18 '12 at 23:54

GUI libraries tend to be inelegant because they address several concerns - input, rendering, and data access - and each of these is heavily influenced by the rest of the program. This means people add abstraction layers to make the GUI reusable across different programs, input devices, and renderers, at the cost of making it more complex to use.
–
KylotanFeb 19 '12 at 15:01

1

Configuring the layout by an artist requires an editor no matter if you're using retained or immediate GUI. This point is completely invalid. Mixing logic and presentation is THE reason you use immediate mode GUI, that's not a flaw, it's what you want, as opposed to the mess that is retained more GUI. You don't even understand that much. The configuration doesn't require globals or extra arguments if the API is well designed. Every downside you mention is solvable in a clean matter. Most importantly though, you're writing a game UI, not Photoshop.
–
dretaJun 22 '14 at 17:19

As someone with five or six years of actual experience with IMGUI on the desktop, I feel obligated to defend it. All of the responses (and the question itself) are based on a very restricted definition of IMGUI, and there seem to be a lot of questionable claims being made.

A lot of it originates from the assumption that an IMGUI library can't retain any data under-the-hood. This isn't true at all. A sophisticated IMGUI library will in fact retain about as much data as an equivalent RMGUI library. The difference is that an IMGUI library mostly caches results, whereas an RMGUI library maintains authoritative state. In IMGUI, the application provides a function that takes the current model state and produces a GUI for that state. This is the authoritative definition of the GUI. The library is responsible for making sure that the on-screen GUI matches the result of that function. This doesn't mean that it needs to fully evaluate that function each frame and fully rebuild the GUI. That's the point of caching.

With this view of IMGUI, you can in fact do advanced layout, and the performance is quite reasonable. You can also retain actual authoritative state if it makes the interface easier. The point of IMGUI is not to make the application retain everything. The application probably doesn't want to have to store the position of every scrollbar and the expanded/collapsed state of every tree node. The point is to be able to procedurally specify the contents of those tree nodes and their very existence.

I would go through and respond to all the IMGUI criticisms point by point, but I don't really understand a lot of them. Most of them seem to stem from some fundamental belief that IMGUI is hackish and RMGUI is well-structured, which I guess stems from the above misunderstandings. There's no inherent reason that IMGUI libraries should have trouble with complexity or have to be littered with globals or mix logic with presentation. I will say that the advantages of IMGUI become less important the more artist-driven your content is, but I'm not sure why it would actually be worse for artist-driven content. (I don't personally use artist-driven content, aside from style sheets for things like colors, font/border sizes, etc., but I don't see why it would be harder to implement than for RMGUI.)

In my mind, IMGUI is unquestionably a good thing. It gives you a much better model for reasoning about your GUI. It becomes much more functional and reactive, and you minimize the amount of mutable state that you have to reason about. On the other hand, implementing a sophisticated IMGUI library is quite a challenge, and definitely more difficult than implementing an equivalent RMGUI library. It's a tradeoff between library complexity and application complexity. If you're planning on building complex apps (or even a lot of simple apps), the tradeoff is very good. If you're doing this for one single app, and it's pretty simple, you'll probably achieve your goal faster with RMGUI.

"There's no inherent reason that IMGUI libraries should have trouble with complexity or have to be littered with globals or mix logic with presentation." It's very hard to see how the usual IMGUI button call, eg. "if Button(x,y) then Action" can not be called mixing logic with presentation. If you don't pass in rendering details to the call then you can't position or style it (except with a global or static), and if you don't process the button logic immediately then it's not really an immediate mode GUI. Could you provide an example to explain?
–
KylotanFeb 26 '12 at 22:04

In OpenGL, when you render a triangle, you don't pass in the transformation matrix, the clip planes, the shader, etc. as parameters to the triangle rendering function. They're part of the context in which the triangle is rendered, and they're specified at a higher level, often hierarchically, for large subsections of the scene. In an IMGUI library, the style information for a widget, its position within the layout tree, etc. can be specified in a similar way. A typical button call in my library is just "if (do_button(context, "OK")) …".
–
TomFeb 27 '12 at 3:59

The OpenGL context is a big source of bugs due to the amount of shared state, and yet even with that API the move has been towards retained mode over the years because immediate mode neither offered the flexibility or the performance required.
–
KylotanFeb 27 '12 at 11:02

If you're concerned about shared state, then why make the context parameter mutable state? Modify it functionally, and you can have all the concurrency you want. As for the flexibility and performance of OpenGL, as I was trying to say above, the shortcomings of one particular IMGUI API don't necessarily imply inherent limitations in the concept. Do you have specific concerns that you believe are inherent limitations, in light of what I've already written in my main answer? I certainly haven't said all there is to say, but you seem hung up on some very limiting assumptions about the concept.
–
TomFeb 27 '12 at 14:52

1

Does a CSS stylesheet also qualify as "one large block of state that is shared"? In any case, I don't know what your particular experience is with IMGUI, but if you're interested in IMGUI approaches that don't suffer from those problems, I'd encourage you to read my answer above and to read through the IMGUI forum on Molly Rocket. It's true there isn't a perfect off-the-shelf IMGUI library that you can download and start using in minutes, but if you're interested in building one, there's information available and people willing to help.
–
TomFeb 27 '12 at 16:55

The GUI library consists of one-shot "RenderButton", "RenderLabel", "RenderTextBox"... functions that you can call to "immediately" render a control (doesn't have to be immediately written to the GPU. Usually its remembered for the current frame and sorted into appropiate batches later).

I would go so far as to say that this doesn't constitute a GUI library. It's just a bunch of rendering functions.

Rendering the GUI is the easiest part of making a GUI. Take a text box for example. I'm going to assume the easiest possible case: a simple single-line entry text box.

RenderTextBox will just draw a text box. It will do some simple text measurement and draw the glyphs on the screen. It will not

Detect that the user has clicked the mouse in the control and move the cursor to the proper location in the string.

Detect that the user double-clicked the mouse in the control and select a block of text.

Detect that the user pressed the "Home" key and move the cursor to the front of the text.

Detect that the user pressed a valid key and modify the string accordingly. If text was selected, the selected portion will be replaced with the inserted text.

I can keep going, but I think my point is clear. If you want to use RenderTextBox to draw a text box, all you will get is a static, non-functional text box. Any actual functionality must be implemented by you.

Doing text measurement and drawing glyphs? That's the easy part of GUI work. GUI stands for "Graphical User Interface". The last two words, where you take input from the user and do something with it, are the hard part.

Game players expect GUI controls to work reasonably. In a PC-style environment (ie: mouse and keyboard), if players see a text box, they expect it to work like a regular text box. They expect to be able to move around in it with arrow keys, jump to the front and end with home and delete, select letters in it, etc.

That means you're going to have to implement all of this UI code. You have to test what control was clicked. You then have to do something based on which control was clicked. You have to have the concept of an "active" control, the one that gets keyboard input. Different controls must respond differently to different types of user interaction.

Basically, you're going to need a TextBoxWindow class.

Let's say you're implementing a game options screen. So you have your different groups of options: gameplay, graphics, sound, controls, etc. So you have a list of different groups on the left side of the screen. Each group brings up a series of controls that the user can manipulate. What happens when the user presses the Tab key?

Well, it depends on what control is active. If one of the group controls is active (was most recently clicked), then it moves to the next group. If instead one of the controls within the group is active, then it moves to the next control in that group. That's how the system is intended to work.

So not only must an active control process keyboard input, it must now farm any unprocessed input to the control that owns it. Either that, or the controls must be in some kind of linked list, so that it can go back and forth. That means there needs to be default behavior in each control.

This sounds more and more like a retained GUI, doesn't it? The only difference is that... you are the one doing the hard work. Using someone elses GUI is supposed to be a way to do less work. But the effort it takes to make a GUI respond as the user expects is non-trivial.

Remember: the UI isn't for you, it's for your users. It's for the player. It should behave as they expect it to. And if it doesn't, then you have failed.

User interfaces usually don't need to be auto-layouted or have resizable windows on-the-fly.

That is a limited and limiting thought.

I could give the spiel about different games having different UI needs, that some games do need resizeable windows and so forth. But there's a much bigger problem.

Your kind of thinking leads to the Gratuitous Space Battles problem.

If you've never played that game, it's a cheap, indie, and very GUI-heavy game. The actual gameplay is all done in the GUI (it's a "setup the units and watch them fight" kind of game). The problem?

THE GUI CANNOT BE RESCALED!

Oh, this is fine if you're using a normal monitor or have normal eyesight. But if you're me, for example, who runs a ridiculously huge monitor (one so big that you have Windows scale up the size of all text so you can actually read it), this is terrible. The text is microscopic. There's no way to change the font size.

Granted, GSB is a GUI-based game, so it's absolutely inexcusable for them. A GUI-lite game can probably get away with it.

Never assume that your user is looking at their game exactly as you are. To do so is folly.

Having a GUI system that can rescale itself to the user's resolution, a truly resolution-independent GUI, requires layout to be part of the GUI system itself. If you're not going to provide that, then your text is going to appear tiny on someone's giant monitor. This is not a good thing.

So I would say that your thinking on this subject is shortsighted. You do need that stuff. You need what retained GUIs provide. Oh, you may not need everything any particular GUI system provides. But you need some of it.

The boilerplate work you need to do to get data to the system is a small price to pay next to having reasonable assurance that the user can interact with controls correctly and that it will resize itself to fit the player's needs.

You might be able to get away with an immediate-mode GUI if you don't take much user input from the player. But that'd be about it.

Please don't take this the wrong way, but may I ask again whether you speak from real-world experience with actually using an immediate GUI approach in a game development (and as I read your opinion: probably replacing it halfway into the development out of frustration ;))? By the way: your pictured functionality of a textbox can be (and is, e.g. in UnityGui.TextField) implemented within in the RenderTextBox class. Also, nothing in the Immediate GUI approach forbids the library designer to use classes for GUI controls, if its appropiate. Just the use would be fundamental different.
–
ImiFeb 18 '12 at 18:50

Other remarks: Yes, I am aware that in some areas any immediate GUI approach can become more "retained"-ish. Also, both concepts could be combined to some degree (e.g. some controls could be drawn in a retained-style fashion, if that makes things easier). I fail to see an argument against a general "Immediate GUI approach" in this, but I would love to hear (preferable non-theoretical) recommendations about where this works and where might be problems. (It works the other way around as well: If you implement realtime user-painted controls - this is some step towards IMGUI)
–
ImiFeb 18 '12 at 20:35

@Immanuel: "By the way: your pictured functionality of a textbox can be (and is, e.g. in UnityGui.TextField) implemented within in the RenderTextBox class." You said that RenderTextBox was a function, not a class. So your division between "immediate mode" and "retained mode" seems to be around where the data is stored, not who's managing it. If so, you need to make that distinction more clear in your question, because it suggests that an "immediate mode GUI" simply draws stuff on the screen. That it can't get input (because that would require persistent state) and such. So which is it?
–
Nicol BolasFeb 18 '12 at 21:39

@ImmanuelScholz: "I fail to see an argument against a general "Immediate GUI approach" in this" I thought I explained it pretty well: someone has to write the code. Someone has to get input, move the cursor around, manage a current control, handle layout, etc. Your description of an "immediate mode GUI" loses all of that, and all of that is important for anything save the most simplistic output-based GUI. So I fail to see your argument that the "immediate GUI approach" has any upsides whatsoever.
–
Nicol BolasFeb 18 '12 at 21:42

1

Sorry, my fault. "...implemented within the RenderTextBox function" (not class). It is a function and does process user input and the library do hold some state (like current focused control). Please, no offense intended, but may I assume that you only know "immediate GUI" from my posting here and neither have looked into IMGUI nor Unity3D gui? I surely did not comprehensible describe what a "immediate GUI" contains in detail here, I just wanted to summarize so we aren't talking about total different things (e.g. to not get confused with "immediate graphic mode").
–
ImiFeb 18 '12 at 21:56

User interfaces usually don't need to be auto-layouted or have resizable windows on-the-fly. Instead, they need to interact very efficiently with always-changing data (like 3d-positions of models in the world)

It depends on the genre and game in question. A good inventory management system is vital for most any RPG. You going to want something that handle layouts for you, supports scroll bars, drag n' drop, tooltips, and other features much easier to implement using Retained GUI.

I can't think of a way to drag n' drop with an immediate GUI that doesn't take a lot juggling or user state saving such as changing the Z buffer temporarily and storing time info to rule out a click that happened to move a bit.

But from a design perspective I have trouble imagining a world without my GUI events. Also an Immediate GUI is not compatible with OOP forcing you to resort to procedural programming.

The simplicity afforded by immediate GUIs comes at the cost of additional complexity in your code that grows rapidly as the complexity required by you UI increases; where as a good retained GUI is more complex originally but scales better. So below a certain point of complexity a immediate GUI is adequate but for most situations I'd rather a retained GUI.

I want to ask you: Does your opinion comes from actual real-world experience or from theoretical thought by looking at both systems? Uhm.. this sounds more harsh than intended, sorry.. What I mean is: I share your opinion and evaluation from a theoretical aspect. IMGUI-like systems most probably lead to less OOPish code (if that is a concern), they have problems with controls that require some state and so on. Its just... normal GUI systems are also very large and complex already by themself, so you have a lot of complexity to add until you even come close to these systems..
–
ImiFeb 18 '12 at 9:25

Immanuel, you don't need to have real-world development experience to know that many games do need auto-layout and resizable windows.
–
KylotanFeb 18 '12 at 21:06

@Immanuel Scholz I admit that I have much much more experience with retained UIs; maintaining one as an OSS projects and having worked with them though my carrier (which admittedly is very young). However, I have used Unity3D's UI system and duplicated it when I moved to XNA. My retained UI was developed because I got tired of copy pasting the same hack from project to project to achieve uniform functionality.
–
ClassicThunderFeb 18 '12 at 21:17

@Kylotan by "resizable windows" I mean that you can actively resize UI elements by dragging (or other means) and by auto-layout that when you resize them, the UI adapts in a "nice" way like displaying scroll bars, allocating multiple lines for button bars and so on. Yes, sometimes I see this in games but its much more common in desktop applications. Beside, I think making good resizable UI's aren't the easiest thing for retained GUI systems either.
–
ImiFeb 18 '12 at 22:05

Resizable windows and autolayout are much the same thing. In retained mode GUIs each control knows its size (whether it changes at runtine or not) and autolayout is basically a recursive size check. Arguably it's more important in games than desktop apps because we expect full screen GUIs in a variety of aspect ratios and on different platforms.
–
KylotanFeb 19 '12 at 0:56

I worked a lot with Unity3D GUI system, which works just as you described. And I must say, I hate it with a passion.

"Immediate" approach works really good in some cases. First, when you only need a couple of buttons and labels - think shooter HUD, for example. Creating them with "retained" approach is not very hard, but its super-easy with UnityGUI. Second case is when you need to stuff lots of controls on screen, but do not really care about layout. Especially when exact controls are only known at runtime. This is not actually useful in any game, but really helpful when making tools running inside Unity editor.

However, any half-complex GUI - for an (MMO)RPG or a strategy game, for example - inevitably turns into a horrible mess of indecipherable code, full of special cases and breaking in tons of ways. When building such a GUI, you usually have pretty specific layout that must work for any resolution and be able to show whatever data you send its way. You need things like, for example, moving some buttons lower to make way for longer text. With "retained" GUI, you can have a control that does that automatically. With "immediate" mode, there are no controls, so you have to do everything explicitly.

Another problem with UnityGUI (not sure if it happens to all immediate mode GUIs) is that, for example, a Button() call works without considering any other GUI calls. So, if one button is on top of another, a click will press both buttons at the same time. This is definitely not what user expects, so this adds another special case to handle.

To live with all this, I've always written my own wrapper around UnityGUI that turns it into a "retained" mode system: controls that store their own state and just call required GUI functions every frame for me.

I can't say that "immediate" mode is definitely worse than "retained", but I certainly feel much better working in "retained" mode.

A while ago IMGUIs also caught my interest, as a test I wrote a couple of tutorials to familiarize myself with the techniques (start here if you're interested, but it's not much more than C#/XNA + the original 'tutorial' here).

I really liked IMGUIs for a short while because the info they display is always up-to-date and correct (since there is no state you always feed to actual data). But in the end I lost interest so normally now I use retained GUIs again.

In the end I thought that the immediateness of the GUI made it harder to separate the GUI from the data and at times I tried to decouple things more by introducing some state, breaking the elegance of IMGUIs. I also don't think that writing code for retained GUIs is more work than writing code for IMGUIs and I like that retained GUIs can be fired and forgot about, and I really like events :).

However, every time someone mentions IMGUIs something inside me wants to try again, and try harder to get it right, because there really is some elegance in there, I'm just not 100% sure if it will always make your job easier. I'd say try it out, maybe try as I did and write some tutorials (I find the best way to learn new techniques is by explaining them to others).

I am going to defend IMGUI here because some of the problems that were listed previously can be solved if you are willing to write the code for it. Additionally if you do write the code for it you then have a robust library that is re-usable and only very rarely requires modification.

loading GUI from schema

perhaps at first it seems that artists do not have a lot of flexablity with IMGUI, this is solved or improved with loading GUI layout from an xml or JSON schema.

menu layer draw calls

this problem is solved with sorting, you should create UI Manager class that sorts the order of draw, I solved this problem in C# XNA with moveable, click-able panels that draw on top of each other. I can post pseudo code if asked.

list boxes

can you draw strings from an array? can you define rectangle areas from the height and width of the fonts of those strings? can you create a ratio of the size of your scroll button by the height of all those rectangles added together? Then you are halfway to creating a listbox with a scrollable button to move up or down. I thought it was going to be hard, but it turned out to be much more simple than I thought. no buffers, no states, no callbacks involved.

separating data from GUI

don't let the individual panels, buttons, boxes hold the data. I know that at my first attempt of IMGUI was immediate but only in the graphic sense. I made the mistake of retaining data on the UI. It was almost a philosophical shift to think differently on where to place and modify data via the UI changes.

GUI from SDKs

this is the limit and functionality of the SDK's code, not yours. I find that SDK UI (Hammer, Unity, UDK) is almost a new language to learn anyway. In fact they are similar to me as Windows.Forms, both are set up for the broadest possibility and therefor have some extra weight in complexity.