Turning Compositing off in the Right Way

I have a dream: a dream of an always composited desktop. There are use cases for using compositing and there are usecases for not using compositing. For the user it is very difficult to know what he currently needs and can do a very bad job at deciding himself. A good example for that is turning compositing off to save some more minutes of battery. I personally doubt that turning compositing off will save battery, but will cause a further drain. How is that possible, you might ask? When compositing is turned off, Plasma starts to change the backgrounds of all SVGs causing in the worst case a re-rendering of all of them. This is costly and will most likely drain more battery than using compositing.

Another example are fullscreen applications. Currently KWin supports unredirecting of fullscreen windows. This means the screen is no longer composited, but the resources, that is the OpenGL context and the effect system is still running. For applications like a web browser or an office suite it is completely useless, while for OpenGL applications like games just unredirction might not be enough. If you have one of those awesome drivers not supporting two OpenGL contexts at the same time, you might see either artefacts or a crashing KWin. For games there is only one proper solution: turn off compositing. The same is true for watching Full-HD videos: often graphics cards are not powerfull enough to do both compositing and GPU accelerated decoding.

Now KWin supports the solution for this: suspending of compositing. Just press Alt+Shift+F12 (or for the more technical users: use a script to change the state through DBus) to suspend compositing. The OpenGL context is removed, the effect system turned down and you have the plain old X desktop. Of course we cannot demand from our users that they know about it and can handle it. Therefore we need a bettter system.

This is what Thomas has been working on lately. First of all he removed the difference between disabled effects and suspended effects. If you disable effects it will just suspend them and the effect system will be in suspended state after a restart. This should make everything more clear to the user. The second part Thomas has been working on is allowing applications to block compositing. This is pretty awesome. Let’s say we want to watch a video in VLC. As soon as you would switch to fullscreen, VLC would set an X property to tell KWin “now please don’t composite”. KWin will suspend compositing and keep the state untill no window blocks compositing. So the GPU is completely free for VLC. Now what is the difference to the unredirection, you may ask? Imagine a user notification would pop up. With unredirection compositing would be started again, causing an ugly flickering and taking away important resources from VLC. With the new solution a notification would not cause a restart of compositing. Everything is still with VLC. On the other hand a web browser would not block compositing as that is not what the user wants. Here we need the complete advantage of a composited system as it’s not a single task such as watching a video or playing a 3D shooter. I really hope that video players, games and Wine pick up our new property and we will also recommend it as an additon to the NETWM specification.

Though the last piece might turn out difficult. While Plasma completely supports being without compositing, the world looks different with the two new Desktop Shells to be released this month. In my humble opinion both are fundamentally flawed concerning the fallback to no compositing. My biggest concern towards GNOME Shell has been from the beginning that it requires OpenGL (I talked about that part with Owen Taylor at GCDS). There is no fallback except GNOME Panel. Which at least looks for me unacceptable to switch the desktop shell just because you want to watch a Full-HD video. Now Canonical did the same fundamental mistake with Unity. It also requires an OpenGL compositor (in that case Compiz). While Compiz nowadays supports non-OpenGL I do not know how good this is and whether Unity supports it. Currently the fallback is also GNOME Panel, in future maybe Unity2D? Plasma on the other hand just needs to switch the rendering of SVGs (which is part of the style) and will lose some functionality, such as thumbnails in taskbar tooltips, Present Windows effect or Desktop Grid. The basic working with the system remains the same. Another big thumbs up for getting the right abstraction layers.

Now with Thomas work I am confident that we will be able to remove the UI to turn on/off compositing in maybe 4.8. It will just not be needed any more. The applications will take care of providing the right user experience to the users. When compositing is needed, it is on, when compositing is bad at the moment, it is off. The users won’t have to care about the state anymore and an ugly hack like unredirection of fullscreen windows can be removed. I am looking forward to such improved ways of desktop compositing 🙂

67 Replies to “Turning Compositing off in the Right Way”

Maybe an option could also be added to application shortcuts so that you can select wther compsiting should be turned off for that application? Also what about adding the option into special window/application settings of KWin?

Basically yes, but in practise no. Windows has an API call for it, so all it needs is to implement that in wine and all games using the API call on windows will be ready. But yes, if we add a new standard we need to implement it in each and every application.

Sorry if i dont get it, but Beat Wolf asked about games and you answered about wine … there is something i dont get.
Additionally i think that this is the wrong aproach to do something that forces every game to change their code.

Looking at the openSUSE games repository, it looks like it has over 200 Linux-based OSS games. And I know that is only a small fraction of the Linux games available (I think the Packman repo had at least that many games that were not also present in the openSUSE repo).

There’s a difference between games and games. We only need changes for games which are demanding on GPU – let’s call it the FarCry category. We don’t have many of those in Linux – KPat is not a problem.

Also there are a LOT of games that are closed (everything from id software, world of goo(in the past), and LOTS of games) and you can’t patch those, yes you can tell them to update the games but in the case of semi-old games(again doom3) most companies don’t even remember having sold those games, they are past stuff.
Rules by application seem nice, but they are very undiscoverable, maybe put a default rule for all menu items which are included in “games category” in kickoff? I think that category is designed by packagers and game developers, so disabling compositing here might be enough.
Also other 3d stuff like google earth, or phun, how much it will be till these implement the api? I think years, so manually disabling should definitely don’t die, maybe it’s use could be minimized with that wine patch and making compositing off default for (fullscreen) games in kickoff.

So is this sort of an API that applications can use or is it a kwin only “feature” ? I remember Windows Vista/7 is doing the same turning off aero when playing a movie in fullscreen or playing a game (at least that’s what i remember, haven’t been using windows for a loooong time). Is there a stable API for such things or the programmer has to make up his own mind how to solve things?

How about dual monitor setups with that aproach? Most of the time i got a fullscreen video on one monitor and everything else on the other. But sometimes i move a window over the fullscreen video to have both visible.

So for me, and some other dual monitor setups it effectly means no compositing if my fullscreen app says: no compositing
As i am running the fullscreen most of the time it reduces to: no compositing at all.

I hope it will be possible to nail the state of compositing state regardless of the applications’ wishes. That is, permanently enable or disable compositing, if one so wishes, despite the performance hit.

Please stop going all gnome on us, removing configuration. Some of us are stuck with buggy drivers (ever heard of GMA500?) and an easy way to enable/disable compositing at will, ignoring application wishes, will always be needed. If you’ll keep the API, and also the shortcut (ctrl-alt-f12), then why remove the configuration option?
One of the things I liked best about KDE (and linux in general) was that I could fine tune it to my hearts desire. Now things are getting more and more windows like, configuration options disappear, and most stuff becomes automatic, with no workaround when automatic fails.

@jbernardo: AFAIK, disabling compositing completely will still be possible, just enabling it always will be removed I think.

@martin: First, sorry if my comment seems rude,I’m just giving my opinions, second: I think forcing people to use automatics it’s not a good option, I don’t think those options do any bad.
There are 2 I really like about your work:
1-mergin disabling and suspending into suspendig
2-making window based disable compisiting for apps that don’t support this api
There are 2 more thing you should do to make everyone happy:
1-Don’t disable the shorcut (alt shift f12)
2-Add “Allow applications to disable compositing” in settings, and enable it by default.
This way everyone is happy, even if drivers are the best in the world and the software workd good everywhere, there will be people that will prefer a few less fps for being able to press alt tab to the desktop from a game and have a compositing desktop to chat with game companions for example. There are a lot of usecases where people won’t want apps to disable compositing, please think about it.

Removing the option to turn on/off composition isnt good idea at all. First, if buggy drivers slow down desktop when compositing is enabled and the user will want to disable it, he couldnt do that easily, and he will have a laggy desktop environment. On my old pc that was the case – radeon 9600 pro didnt made all of the animations smooth, so at some points i needed to disable it just to make desktop snappier. Nothing horrible, but some of the users would rather prefer no effects at all rather than having them choppy. Even in Windows you can disable compositing.
On the other side, speaking of desktop in which you never need to disable compositing, im already having it. Since last year when i bought new pc i never used kde without compositing, altough the option to disable it manually should always be here, sometimes when you update drivers compositing can crash your xserver because of buggy driver update and without having option to disable it, that would be impossible to investigate the problem.

Of course KWin will only allow compositing on systems which support it. But it is not our fault if users want to run the latest software on old hardware. Some day we have to start thinking about not supporting legacy hardware. This day will come at least with Wayland – and we will always keep the Alt+Shift+F12 key, just not expose it in the UI anymore.

I use Lenovo T400, Intel Integrated Graphics 4500MHD. Linux kernel 2.6.37, actual x.org and intel driver. Not so legacy system, but with compositing it is unacceptably slow. Konsole is slow, kate with larger file is slow and firefox is sluggish. Without compositing it is snappy.

What do you mean by “kwin will only allow compositing on system that support it”, I don’t think kwin should be the one making the decitions, unless it’s told to, I’m the owner of the system and I decide if I can bare slow compositing on old pc because it looks nice.

If KDE is run remotely via an XDMCP session it performance is bad, really really bad. While using Gnome (2.32), the performance is fine and KDE applications like kontact, can be used normally. I think that this has to do with compositing. If so, is it possible for kwin to take remote XDMCP sessions into account, and switch compositing off when run over XDMCP?

First of all we would need to know if turning compositing off helps. In general it’s a damp stupid idea to use compositing on a remote desktop. The question is how kwin should now that the screen is forwarded to another system.

What happens when an application requests to turn off compositing and then crashes? Under Windows compositing is re-enabled then but there are a few cases which make that automatic procedure that looks great on paper a mess in reality.

For example under Windows some web browser plugins always disable compositing (QuickTime and QuakeLive come to mind). So sometimes they crash but since Firefox itself continues to run, compositing is off until FF is closed and restarted.
I really don’t know about the use cases of the average KWin user but in my case compositing does not greatly affect the performance of QuakeLive (Linux version). The average video also plays just fine and I rather have smoothly fading playback controls, OSDs, etc.
What, however, does affect performance on my system is GPU-accelerated HD video decoding along with compositing.
So this is really a corner case for which manual switching is enough and I don’t want compositing disabled for any other case automatically whenever some random application decides it’s OK.

What happens when an application requests to turn off compositing and then crashes? Under Windows compositing is re-enabled then but there are a few cases which make that automatic procedure that looks great on paper a mess in reality.

The block is implemented through an X property. If the window is withdrawn (happens at crash) the block is withdrawn.

For example under Windows some web browser plugins always disable compositing (QuickTime and QuakeLive come to mind). So sometimes they crash but since Firefox itself continues to run, compositing is off until FF is closed and restarted.

That depends on how Firefox implements it, but in general I would say that Firefox should never set the property.

What, however, does affect performance on my system is GPU-accelerated HD video decoding along with compositing.
So this is really a corner case for which manual switching is enough

I don’t think that watching Full-HD videos is a corner case any more and I don’t think it is acceptable for the average user to know that there is compositing it all. For them it’s just “KDE/Linux is slow”.

There still is the window rules system which lets you overwrite the application’s wish. So we give for the unexperienced users the overall best choice but keep the ability to control every aspect for the advanced users.

Watching HD video in a web browser is pretty common nowadays. Flash finally got support for hardware-accelerated video on Linux and HD HTML5 video is becoming more and more popular. So I would expect that web browsers would routinely need to set the property since it would routinely need to display HD video.

That depends on how Firefox implements it, but in general I would say that Firefox should never set the property.

Well, Firefox itself doesn’t. The plugins do and Quake Live is a full-blown 3D fullscreen game just as QuickTime is a full-blown video player – both are just embedded in a web browser for interaction with a web site.

I don’t think that watching Full-HD videos is a corner case any more

I meant in the grand scope of things I do on my PC, it’s a corner case. Working GPU-decoding of videos is another corner case, sadly. My GPU is also a few years old already.

Maybe it’s possible to create a generic way to check for the performance and then decide what to do.
If GPU decoding and compositing clash, act according to a pre-set rule of either disabling compositing or or switch from GPU decoding to CPU decoding.
In my case I’d rather see the CPU take over decoding the video than to have the negative side effects of disabling compositing (ugly desktop, ugly fullscreen controls, flickering).

“A good example for that is turning compositing off to save some more minutes of battery. I personally doubt that turning compositing off will save battery, but will cause a further drain. How is that possible, you might ask? When compositing is turned off, Plasma starts to change the backgrounds of all SVGs causing in the worst case a re-rendering of all of them. This is costly and will most likely drain more battery than using compositing.”

Dude, you are a fucking idiot. Kill yourself.

To the KDE project core staff: get a better developer for KWin, please. Martin Graesslin has been proving his uselessness and stupidity for too much time.

That was definitely rude, I also apologize if I seem to don’t valorize yout work, you do a lot of awesome stuff, but as everyone, you can make mistakes, and thats why we are here, to help discuss these stuff, and decide who is the one making the mistake.
About the original problem, I think some real life benchmarks should be done, if the gpu is very power hungry and even better in the case of hybrid gpu, then it might save some minutes. This is an open discussion, and I think people should be allowed to disable and enable compositing whenever they feel like it, tough this can be perfectly made, just adding an additional rule to powerdevil profile that makes a call to that api function (The default should be not disable, but powerdevil can give the option)

Yes thanks Martin, for leaving that post up there so the rest of us can see the
sort people you have to deal with. Your reply was spot on for that particular case
and i really appreciate the time you have taken to reply to all the other legitimate
queries. Fantastic work!

Well since you are an “expert” in power management, kwin internals and plasma internals I’d like you to share with the rest of us your technical reasons disproving
martin’s assertion. In advance I think you can’t. I think you said what you said because intuitively you believe anything done in the gpu uses a lot of power. FYI Aristotle thought that every body tended naturally to be at rest because he saw that every body in motion eventually stopped. Later on Newton postulated the existence of a non-conservative force called friction along with his laws of motion, which are still useful in spite of the discovery of the special and general relativity. Since you’re *clearly* not an idiot you will get the message behind this parable.

This doesn’t help for things like games or Full-HD videos as there the impact is the compositing and not the effects. And turning off the cube effect if cube is not used is rather useless. The only effect where it might make sense is for example blur when running out of battery. But disabling blur causes Plasma to render a different set of SVGs, so whether there would be an advantage is not possible to be said without very good performance testing (which in case of blur depends much on hardware and drivers).

First, why would disabling blur cause plasma to render a different set of svgs for anything other than the panel and some pop-ups?

Second, disabling the cube makes a big difference if you have it set as your default desktop switcher. You seem to be suggesting people just shouldn’t change virtual desktop when their system is under heavy load or low on batteries, which seems to be a major loss of functionality. The same is true for if you use, for instance, shift switcher for your default application switcher, wobbly windows when dragging windows, darkening non-focused windows, and the slide back effect when changing window focus. So no, these effects do not take resources or hurt the battery life when they are not being used, the problem is that many of the effects will be used while doing ordinary tasks on the computer.

I know it doesn’t directly apply to full-screen video, but I thought the consensus on the mailing list was to implement this option, and it is related to the general topic.

It really depends on what the effect is doing and you need to know the implementation of the effect (which I know). Consider darkening of non-focused windows: it is just modifying the rendering of the window. That is the same code is executed whether the effect is activated or not. Turning it off, will have no impact. Now concerning e.g. Shift Switch I think that the change in functionality (switching from shift to classic) is more a problem to the usability than some five seconds on battery. Slide-back and cube animation (not cube) are the only ones where it may make sense to consider turning it off. As those are pure animations. Everything else like Present Windows, Desktop Grid or Cube does not make sense.

I did not see any consensus on the mailing list that we need to turn off the effects. At least not from developers knowing about how the effect and compositor in fact interact. I think you need this knowledge to really understand what is going on and what makes sense.

Still people might prefer that 5 seconds of battery life, it might not make sense for you, and I respect that, but it makes sense for a lot of people, so leaving the option is still a good idea, even if it is just 5 seconds, and even if it’s just placebo effect.

After reading all this and specially the discussion, i predict this to become a disaster. I can also predict that any discussion on the subject will be useless… Thats going to be fun 🙁

Hoping for non kde apps to implement this is wishfull thinking, and i can name way more than 5 games from the top of my head that run under linux. And there are many non open source ones too.
The answer about gpus with driver problems is also bad.
I hoped the discussion in the mailing list gave a good result, but it doesnt seems so.

I hoped the discussion in the mailing list gave a good result, but it doesnt seems so.

Of course it did. This blog post is in fact a direct consequence of the discussion of the mailing list: we need to have a proper infrastructure in place to go the way. With these patches we have the infrastructure now we just need to get people use it.

And since this is KDE there is an option to disable automatic disabling (:P) of compositing since I have four monitors and I tend to play with one only and my graphics cards can handle this, I would never want compositing to be turned off 🙂

This seems like an excellent idea. I look forward to see it in action 🙂

To the naysayers: kwin will have the API to disable compositing, so you’ll always be able to do it. So what’s the problem?
Also, if later you have tried this system and you find that you want to always enable or always disable compositing, you can always file a request or patch kwin — It’s probably a one-liner to ignore app requests.

I agree with all the proposed changes (thank you for implementing this and for continuously improving kwin!!!) except the removal of the “Enable desktop effects” check box in the GUI. Sometimes (luckily not often) it happens that compositing becomes so slow that it disables itself and then I am happy to enable it again in the systemsettings (I forget the keyboard shortcut, so I go to systemsettings to reenable compositing) after which compositing runs fluently again. Back in the days I chose to use KDE 3.x instead of Gnome because whenever I wanted to tweak my desktop there was an option in the GUI to do it instead of having to change an obscure option in an obscure configuration file located in an obscure location (or in the obscure gconf-editor; someone will create a kconf-editor if options keep disappearing from the GUI ;-). In my opinion, the best would be the possibility to choose between “Disable desktop effects”, “Always enable desktop effects” and “Smart usage of desktop effects” (the latter being your current proposal). In the future people will have hardware which will run smoothly with compositing turned always on (even when watching videos or playing games fullscreen), so “Always enable desktop effects” will be the ideal choice for them. People with lousy hardware will choose “Disable desktop effects” and the others will like “Smart usage of desktop effects”. Back in the days I read somewhere in a linux website or forum the sentence “whenever you make any assumption about what hardware or what configuration the user has, there will always be a user who uses hardware that breaks your assumption”. You should keep this truth in mind when designing your software.

You say that the average user should not be aware that there is such a thing as compositing and I agree with that, but I wonder: how is that realizable when (1) compositing from time to time gets so slow that it gets disabled (and the user experience is: “suddenly the desktop looks so different and why does all of a sudden ‘present windows’ not work anymore? kde/linux sux…”), (2) the uncomposited notifications that appear when you are watching a video fullscreen look different than the composited notifications in other situations (“that notification looks so ugly when watching stuff fullscreen; kde/linux sux…”), (3) there is no compositing on a remote desktop (“why does my desktop look different when I access it remotely? kde/linux sux…”)?

Just a short note on compositing and battery usage: I have a shitty NVIDIA GPU in my laptop that stays at it’s highest power use when compositing is on but can go to 2D mode when compositing is off. That’s a difference of >5W. That’s >25% shorter runtime on battery + the shitty NVIDIA GPU gets very hot so that I wouldn’t be able to keep the laptop on top of my lap…

When you say that there may be a increase in power consumption with composite turned off, what support do you have for that? I actually have the oposite case with my laptop. Lastest free radeon driver. When composite is on, I have at least 60 wakeups/second coming from the driver (measured with powertop) on idle and an increased estimate of power consumption. I have not tested with compiz recently to compare but a year ago the situation seemed to be ubiquitous. Is there a way to properly test this and optimize kwin to not stress the driver on idle? If so, how can I help?

1. For powersaving one should consider to use the XRender backend (completely spares the pixmap -> texture conversion. good if you face high frequent updates)
In general compositing should save you some battery, since you’ll unlikely have active server backbuffering on todays Xorg – the compositor does the job anyway (and does it better. backbuffering was designed for low memory chips)
Without server side backbuffering, everytime a region of a client becomes unobscured (you moveresize another client on top) it’s called to repaint itself for that region. Depending on how efficient the client does this, it can be “quite ok” to “horribly expensive”. (Turn off the compositor, select a rather expensive Gtk+ style like orta and drag a window across such gtk+ window to _see_ what i mean) and you’ll pay the server roundtrip anyway (and on /every/ pixel row/column)

Of course fancy things like transparency, shadows, *blurring* etc. all quite stress the GPU and -depending on the driver- take more or less power. But the GPU can be very efficient as well
(PowerVR should currently be the king of the hill, but they do afaik only power ARM devices) 🙁

Oh, and Martin is perfectly right about plasma.
The svg rendering on theme changing (including parsing the xml, do the svg rendering, pixmap allocation, caching & a relayout before the complete edsktop update) /is/ expensive (on the CPU and then the GPU), so turning off compositing “to save some more minutes of battery” is pretty much the worst thing one could do 😉

2. Users of the closed source nvidia drivers should, at some point, consider to try “less /usr/share/doc/nvidia/README” (it’s a large doc, maybe do not exactly use less. does okular have a nice ascii text read mode?)
a) you can not only OVER- but also UNDERclock your GPU
b) you can (at least for most mobile chips) enforce a specific performance mode (look out for the powermizer stuff)

3. multiscreen support in kwin is stepchilded anyway. any interested developer would certainly be more then welcome.
however there’s either compositing or there is not. both at the same time requires two instances of kwin, ie. a multihead setup.
The rules however allow you to downforce the flag and ideally there will be a way to configure the blocking related to fullscreen windows (ie. you could say: “every fullscreen window shall block compositing, but not ” – we may need a “fullscreen” match in the rules dialog)

4. @taramnz – i do not know where you live, so i do unfortunately not have a commercial link, but you desperately want this:http://en.wikipedia.org/wiki/Alprazolam
Once you got that under control, you can provide patches to improve kwin, demonstrate your advanced skills and one day take over maintainership – what btw. likely takes more than the inability to configure your GPU.

Would it be better in this case for the application to send a ClientMessage to the root window with some information on how the window wants to be composited, eg

“Please turn compositing off”
“Please unredirect me”
“Please turn compositing down (implementation is vague here, but I guess you could swap to XRender, or turn off heavy postprocessing such as blur and the like)

I am not sure if a ClientMessage is sufficient. The blocking property allows a window to block compositing e.g. only if going to fullscreen and withdrawing the property when returning to normal state. Also I don’t see how with a ClientMessage it could survive a restart of the compositor or getting a nice overview of what windows currently block compositing.

Concerning a more fine grained approach is probably not a good idea as I doubt that most devs know the exact difference between unredirected and (in case of KWin) suspended compositing.

There’d be no big deal to turn the bool into a flag but the client should not care about “how the window wants to be composited” – it should just know: here i am, i run the show, i’m expensive, please turn expensive stuff off cause it’s pointless anyway.

Blocking compositing for “normal” clients which are esp. not fullscreen but just too expensive (for your particular system) is rather a job for the rules. (so you don’t have to know & press the shortcut whenever you launch gl client xyz because it’s too slow otherwise or mesa will crash KWin)

I may be missing something, but why are having compositing on and playing an HD video incompatible? Is that always the case? I play complex 3D games on my desktop with (metacity’s) xrender compositing, and my current experience with gnome shell is that it doesn’t slow down my 3D apps at all. So can’t we just leave opengl compositing on and forget it?