WebRender newsletter #43

WebRender is a GPU based 2D rendering engine for the web written in Rust, currently powering Mozilla’s research web browser servo and on its way to becoming Firefox‘s rendering engine.

A week in Toronto

The gfx team got together in Mozilla’s Toronto office last week. These gatherings are very valuable since the team is spread over many timezones (in no particular order, there are graphics folks in Canada, Japan, France, various parts of the US, Germany, England, Australia and New Zealand).

It was an intense week, filled with technical discussions and planning. I’ll go over some of them below:

New render task graph

Nical continues investigating a more powerful and expressive render task graph for WebRender. The work is happening in the toy render-graph repository and during the week a lot task scheduling and texture memory allocation strategies were discussed. It’s an interesting and open problem space where various trade-offs will play out differently on different platforms.
One of the things that came out of these discussions is the need for tools to understand the effects of the different graph scheduling and allocation strategies, and help debugging their effects and bugs once we get to integrating a new system into WebRender. As a result, Nical started building a standalone command-line interface for the experimental task graph and generate SVG visualizations.

Battery life

So far our experimentation has showed that energy consumption in WebRender (as well as Firefox’s painting and compositing architecture) is strongly correlated with the amount of pixels that are manipulated. In other word, it is dominated by memory bandwidth which is stressed by high screen resolutions. This is perhaps no surprise for someone having worked with graphics rendering systems, but what’s nice with this observation is that it gives us a very simple metric to measure and build optimizations and heuristics around.

Avenues for improvements in power consumption therefore include Doug’s work with document splitting and Markus’s work on better integration with MacOS’s compositing window manager with the Core Animation API. No need to tell the window manager to redraw the whole window when only the upper part changed (the document containing the browser’s UI for example).
The browser can break the window up into several surfaces and let the window manager composite them together which saves some work and potentially substantial amounts of memory bandwidth.
On Windows the same can be achieved with the Direct Composition API. On Linux with Wayland we could use sub-surfaces although in our preliminary investigation we couldn’t find a reliable/portable way to obtain the composited content for the whole browser window which is important for our testing infrastructure and other browser functionalities. Only recently did Android introduce similar functionalities with the SurfaceControl API.

We made some short and long term plans around the theme of better integration with the various window manager APIs. This is an area where I hope to see WebRender improve a lot this year.

Fission

Ryan gave us an overview of the architecture and progress of project Fission, which he has been involved with for some time. The goal of the project is to further isolate content from different origins by dramatically increasing the amount of content processes. There are several challenging aspects to this. Reducing the per-process memory overhead is an important one as we really want to remain better than Chrome overall memory usage. WebRender actually helps in this area as it moves most of the rendering out of content processes. There are also fun (with some notion of “fun”) edge cases to deal with such as page from domain A nested into iframe of domain B nested into iframe of domain A, and what this type of sandwichery implies in terms of processes, communication and what should happen when a CSS filter is applied to that kind of stack.

Fun stuff.

WebRender and software rendering

There will always be hardware and driver configurations that are too old, too buggy, or both, for us to support with WebRender using the GPU. For some time Firefox will fall back to the pre-WebRender architecture, but we’ll eventually want to get to a point where we phase out this legacy code while still work for all of our users. So WebRender needs some way to work when GPU doesn’t.

We discussed several avenues for this, one of which being to leverage WebRender’s OpenGL implementation with a software emulation such as SwiftShader. It’s unclear at his point whether or not we’ll be able to get acceptable performance this way, but Glenn’s early experiments show that there are a lot of low hanging fruits to pick and optimize such a software implementation, hopefully to the point where it provides a good user experience.

Other options include dedicated CPU rendering backend which we could safely expect to get to at least Firefox’s current software rendering performance, at the expense of more engineering resources.

Removing code

As WebRender replaces Firefox’s current rendering architecture, we’ll be able to remove large amounts of fairly complex code in the gfx and layout modules, which is an exciting prospect. We discussed how much we can simplify and at which stages of WebRender’s progressive rollout.

WebGPU status

Kvark gave us an update on the status of the WebGPU specification effort. Things are coming along in a lot of areas, although the binding model and shader format are still very much under debate. Apple proposes to introduce a brand new shading language called WebHLSL while Mozilla and Google want a bytecode format based on a safe subset of SPIRV (the Khronos group’s shader bytecode standard used in Vulkan OpenCL, and OpenGL through an extension). Having both a bytecode and high-level language is also on the table although fierce debates continue around the merits of introducing a new language instead of using and/or extending GLSL, already used in WebGL.

From what I have seen of the already-agreed-upon areas so far, WebGPU’s specification is shaping up to be a very nice and modern API, leaning towards Metal in terms of level of abstraction. I’m hopeful that it’ll eventually be extended into providing some of Vulkan’s lower level controls for performance.

Display list improvements

Display lists can be quite large and costly to process. Gankro is working on compression opportunities to reduce the IPC overhead, and Glenn presented plans to move some of the interning infrastructure to the API endpoints so as to to send deltas instead of the entire display lists at each layout update, further reducing IPC cost.

Possibly. So far Markus concentrated his implementation effort on the part where Firefox creates the main window and its OpenGL context (getting this right is actually harder than it sounds). This probably has to be done in Gecko’s C++ widget handling code, but the part where WebRender will add/remove/update native OS compositor surfaces should probably be done with planeshift. We aren’t there yet, though.

There are different areas of the code that can go away at different stages of the process. We have a number of compositing and painting backends. The software layers and painting backends will be the last to go most likely, while D2D painting, D3D11 compositing and OpenGL compositing can probably be removed earlier.

There is also a bunch of complexity that goes into abstracting over these various backends, so the less of them we have, the simpler we can make the abstractions.

That sounds nice! I guess it would be optimal to test against both, kwin and mutter?
Don’t know about kwin, but in mutter there are several open issues around subsurfaces and I wouldn’t be suprised if more pop up as soon as they get used more extensively.

> On Linux with Wayland we could use sub-surfaces although in our preliminary investigation
> we couldn’t find a reliable/portable way to obtain the composited content for the whole browser
> window which is important for our testing infrastructure and other browser functionalities.

Can you elaborate on this please. Is it a limitation of existing Wayland features?

I don’t have all of that information handy and wasn’t involved with the investigation but IIRC, wayland doesn’t provide any API/protocol to give you the composited result of a window built upon several subsurfaces.

I don’t know the status of wayland + webrender. Most of the work revolves around using different APIs to talk to the GL context (EGL/GLX), shared textures, how to present frames to the window manager, etc and cattering to each API’s constraints and limitations. Lots of low level details.
Vulkan + wayland is absolutely unexplored territory at this point, but whenever the vulkan work gets shippable (maybe in a year or so, I really don’t know), I suppose that the wayland side of it will be similar to the necessary glue to go from GLX to EGL.