Creating devices with multiple UI processes using Wayland

In the early days of devices with a user interface, there was a single process that ran the user interface component. This was a nice way to separate out the user interface from the logic controlling the device and to ensure responsiveness for the end user. As hardware costs fell while simultaneously becoming more capable, we found more things for our devices to do. The user interface needed to be able to handle many more things, including asynchronous events and presenting information coming in from multiple places. Think of the transition from simple phones doing SMS, to feature phones and now smartphones which are more powerful than PCs of a decade ago. To run the whole UI of a smartphone in a single process would be considered madness today. Why not use similar techniques for building complex device user interfaces?

Devices with a multi-process user interface have several benefits:

Stability and robustness: By splitting your application into multiple functional units you maximize the stability of each application and if one application crashes it will not affect the other applications.

Efficient development: Separate teams can work on each application with lower risk of breaking something that the other team is working on.

Security: You can run applications with reduced privileges meaning that they can’t influence or access each other.

3rd party application development: You can easily enable your device to be opened up to 3rd party app development. You can even mix and match applications from different UI toolkits.

Why Wayland?

Having made the decision to have multiple process for the user interface display, the next technical problem to solve is getting the outputs of each process onto the display. Historically the approach has been to use X and write your own window manager.

The modern approach is to use Wayland, which is a simpler replacement for X.

It’s easy to develop and maintain

It’s a modern approach to window management designed with today’s graphical hardware in mind

The protocol is easily extendable

Wayland is the new industry standard for window management—it’s a common protocol that can be used by everyone, and is extended by everyone

With Wayland, the compositor is responsible for arranging and displaying each client’s content or output on the screen.

Wayland in Qt

Earlier versions of Qt provide the Qt Window System (QWS), which shares some of the design goals of Wayland. However, QWS only supports Qt-based clients and its architecture is not suited to modern OpenGL-based hardware. With Qt 5 we therefore started working on Wayland. Running Qt applications as Wayland clients has been officially supported since Qt 5.4.

Qt Wayland Compositor API

With Qt 5.8 we have released the Qt Wayland Compositor API, which allows you to create your own Wayland compositor. We give you all the tools you need to create a compositor in a simple way. You can easily define the UI and UX of your compositor with the power and simplicity of QML. If you still need or want to use C++, that’s also possible. There are also many other benefits of using the Qt Wayland Compositor API:

Creating multi-screen devices is simple

High abstraction level API means less lines of code to test and debug

Support for easily writing your own Wayland extensions—Allowing easy communication between clients and the compositor. E.g. you could write a protocol extension for telling clients to switch to night mode.

Built-in support for several shell extensions including XDG shell, WL Shell and IVI application allowing to connect to clients using any of these extensions

I hope this has piqued some interest in creating multi-process devices with the Qt Wayland Compositor API. To see for yourself how easily it can be done, check out the 10-minute tutorial below.

9 comments

Wayland is a technology designed for Linux. And while Android is built on Linux, it’s got its own display server called SurfaceFlinger, which is sort of doing the same thing a Wayland compositor would do, so Wayland doesn’t really have a place there.

The Qt Wayland Compositor API is most relevant on embedded linux devices where your other options are to run Weston, X, or just run a single process in full screen mode.

Thank you for the blog post. The tutorial doesn’t show /how/ to set up a dev env and new project to create a compositor. Can some information be shared about this? The online documentation does not seem to help.