ChrisLord.nethttp://chrislord.net
Wed, 20 May 2015 11:35:24 +0000en-UShourly1http://wordpress.org/?v=4.2.3Web Navigation Transitionshttp://chrislord.net/index.php/2015/04/24/web-navigation-transitions/
http://chrislord.net/index.php/2015/04/24/web-navigation-transitions/#commentsFri, 24 Apr 2015 09:26:00 +0000http://chrislord.net/?p=273Continue reading →]]>Wow, so it’s been over a year since I last blogged. Lots has happened in that time, but I suppose that’s a subject for another post. I’d like to write a bit about something I’ve been working on for the last week or so. You may have seen Google’s proposal for navigation transitions, and if not, I suggest reading the spec and watching the demonstration. This is something that I’ve thought about for a while previously, but never put into words. After reading Google’s proposal, I fear that it’s quite complex both to implement and to author, so this pushed me both to document my idea, and to implement a proof-of-concept.

I think Google’s proposal is based on Android’s Activity Transitions, and due to Android UI’s very different display model, I don’t think this maps well to the web. Just my opinion though, and I’d be interested in hearing peoples’ thoughts. What follows is my alternative proposal. If you like, you can just jump straight to a demo, or view the source. Note that the demo currently only works in Gecko-based browsers – this is mostly because I suck, but also because other browsers have slightly inscrutable behaviour when it comes to adding stylesheets to a document. This is likely fixable, patches are most welcome.

Navigation Transitions specification proposal

Abstract

An API will be suggested that will allow transitions to be performed between page navigations, requiring only CSS. It is intended for the API to be flexible enough to allow for animations on different pages to be performed in synchronisation, and for particular transition state to be selected on without it being necessary to interject with JavaScript.

Proposed API

Navigation transitions will be specified within a specialised stylesheet. These stylesheets will be included in the document as new link rel types. Transitions can be specified for entering and exiting the document. When the document is ready to transition, these stylesheets will be applied for the specified duration, after which they will stop applying.

When navigating to a new page, the current page’s ‘transition-exit‘ stylesheet will be referenced, and the new page’s ‘transition-enter‘ stylesheet will be referenced.

When navigation is operating in a backwards direction, by the user pressing the back button in browser chrome, or when initiated from JavaScript via manipulation of the location or history objects, animations will be run in reverse. That is, the current page’s ‘transition-enter‘ stylesheet will be referenced, and animations will run in reverse, and the old page’s ‘transition-exit‘ stylesheet will be referenced, and those animations also run in reverse.

[Update]

Anne van Kesteren suggests that forcing this to be a separate stylesheet and putting the duration information in the tag is not desirable, and that it would be nicer to expose this as a media query, with the duration information available in an @-rule. Something like this:

I think this would indeed be nicer, though I think the exact naming might need some work.

Transitioning

When a navigation is initiated, the old page will stay at its current position and the new page will be overlaid over the old page, but hidden. Once the new page has finished loading it will be unhidden, the old page’s ‘transition-exit‘ stylesheet will be applied and the new page’s ‘transition-enter’ stylesheet will be applied, for the specified durations of each stylesheet.

When navigating backwards, the CSS animations timeline will be reversed. This will have the effect of modifying the meaning of animation-direction like so:

and this will also alter the start time of the animation, depending on the declared total duration of the transition. For example, if a navigation stylesheet is declared to last 0.5s and an animation has a duration of 0.25s, when navigating backwards, that animation will effectively have an animation-delay of 0.25s and run in reverse. Similarly, if it already had an animation-delay of 0.1s, the animation-delay going backwards would become 0.15s, to reflect the time when the animation would have ended.

Layer ordering will also be reversed when navigating backwards, that is, the page being navigated from will appear on top of the page being navigated backwards to.

Signals

When a transition starts, a ‘navigation-transition-start‘ NavigationTransitionEvent will be fired on the destination page. When this event is fired, the document will have had the applicable stylesheet applied and it will be visible, but will not yet have been painted on the screen since the stylesheet was applied. When the navigation transition duration is met, a ‘navigation-transition-end‘ will be fired on the destination page. These signals can be used, amongst other things, to tidy up state and to initialise state. They can also be used to modify the DOM before the transition begins, allowing for customising the transition based on request data.

JavaScript execution could potentially cause a navigation transition to run indefinitely, it is left to the user agent’s general purpose JavaScript hang detection to mitigate this circumstance.

Considerations and limitations

Navigation transitions will not be applied if the new page does not finish loading within 1.5 seconds of its first paint. This can be mitigated by pre-loading documents, or by the use of service workers.

Stylesheet application duration will be timed from the first render after the stylesheets are applied. This should either synchronise exactly with CSS animation/transition timing, or it should be longer, but it should never be shorter.

Authors should be aware that using transitions will temporarily increase the memory footprint of their application during transitions. This can be mitigated by clear separation of UI and data, and/or by using JavaScript to manipulate the document and state when navigating to avoid keeping unused resources alive.

Navigation transitions will only be applied if both the navigating document has an exit transition and the target document has an enter transition. Similarly, when navigating backwards, the navigating document must have an enter transition and the target document must have an exit transition. Both documents must be on the same origin, or transitions will not apply. The exception to these rules is the first document load of the navigator. In this case, the enter transition will apply if all prior considerations are met.

Default transitions

It is possible for the user agent to specify default transitions, so that navigation within a particular origin will always include navigation transitions unless they are explicitly disabled by that origin. This can be done by specifying navigation transition stylesheets with no href attribute, or that have an empty href attribute.

Note that specifying default transitions in all situations may not be desirable due to the differing loading characteristics of pages on the web at large.

It is suggested that default transition stylesheets may be specified by extending the iframe element with custom ‘default-transition-enter‘ and ‘default-transition-exit‘ attributes.

I believe that this proposal is easier to understand and use for simpler transitions than Google’s, however it becomes harder to express animations where one element is transitioning to a new position/size in a new page, and it’s also impossible to interleave contents between the two pages (as the pages will always draw separately, in the predefined order). I don’t believe this last limitation is a big issue, however, and I don’t think the cognitive load required to craft such a transition is considerably higher. In fact, you can see it demonstrated by visiting this link in a Gecko-based browser (recommended viewing in responsive design mode Ctrl+Shift+m).

I would love to hear peoples’ thoughts on this. Am I actually just totally wrong, and Google’s proposal is superior? Are there huge limitations in this proposal that I’ve not considered? Are there security implications I’ve not considered? It’s highly likely that parts of all of these are true and I’d love to hear why. You can view the source for the examples in your browser’s developer tools, but if you’d like a way to check it out more easily and suggest changes, you can also view the git source repository.

]]>http://chrislord.net/index.php/2015/04/24/web-navigation-transitions/feed/6Linking CSS properties with scroll position: A proposalhttp://chrislord.net/index.php/2013/12/16/linking-css-properties-with-scroll-position-a-proposal/
http://chrislord.net/index.php/2013/12/16/linking-css-properties-with-scroll-position-a-proposal/#commentsMon, 16 Dec 2013 11:20:33 +0000http://chrislord.net/?p=255Continue reading →]]>As I, and many others have written before, on mobile, rendering/processing of JS is done asynchronously to responding to the user scrolling, so that we can maintain touch response and screen update. We basically have no chance of consistently hitting 60fps if we don’t do this (and you can witness what happens if you don’t by running desktop Firefox (for now)). This does mean, however, that you end up with bugs like this, where people respond in JavaScript to the scroll position changing and end up with jerky animation because there are no guarantees about the frequency or timeliness of scroll position updates. It also means that neat parallax sites like this can’t be done in quite the same way on mobile. Although this is currently only a problem on mobile, this will eventually affect desktop too. I believe that Internet Explorer already uses asynchronous composition on the desktop, and I think that’s the way we’re going in Firefox too. It’d be great to have a solution for this problem first.

It’s obvious that we could do with a way of declaring a link between a CSS property and the scroll position. My immediate thought is to do this via CSS. I had this idea for a syntax:

This would work quite similarly to standard transitions, where a limited number of properties would be supported, and perhaps their interpolation could be defined in the same way too. Relative scroll position is 0px when the scroll position of the particular axis matches the element’s offset position. This would lead to declarations like this:

This would define a transition that would grow and fade in an element as the user scrolled it towards 100px down the page, then shrink and fade out as you scrolled beyond that point.

But then Paul Rouget made me aware that Anthony Ricaud had the same idea, but instead of this slightly arcane syntax, to tie it to CSS animation keyframes. I think this is more easily implemented (at least in Firefox’s case), more flexible and more easily expressed by designers too. Much like transitions and animations, these need not be mutually exclusive though, I suppose (though the interactions between them might mean as a platform developer, it’d be in my best interests to suggest that they should :)).

I’m not aware of any proposal of this suggestion, so I’ll describe the syntax that I would expect. I think it should inherit from the CSS animation spec, but prefix the animation-* properties with scroll-. Instead of animation-duration, you would have scroll-animation-bounds. scroll-animation-bounds would describe a vector, the distance along which would determine the position of the animation. Imagine that this vector was actually a plane, that extended infinitely, perpendicular to its direction of travel; your distance along the vector is unaffected by your distance to the vector. In other words, if you had a scroll-animation-bounds that described a line going straight down, your horizontal scroll position wouldn’t affect the animation. Animation keyframes would be defined in the exact same way.

[Edit] Paul Rouget makes the suggestion that rather than having a prefixed copy of animation, that a new property be introduced, animation-controller, of which the default would be time, but a new option could be scroll. We would still need an equivalent to duration, so I would re-purpose my above-suggested property as animation-scroll-bounds.

What do people think about either of these suggestions? I’d love to hear some conversation/suggestions/criticisms in the comments, after which perhaps I can submit a revised proposal and begin an implementation.

]]>http://chrislord.net/index.php/2013/12/16/linking-css-properties-with-scroll-position-a-proposal/feed/4Efficient animation for games on the (mobile) webhttp://chrislord.net/index.php/2013/11/29/efficient-animation-for-games-on-the-mobile-web/
http://chrislord.net/index.php/2013/11/29/efficient-animation-for-games-on-the-mobile-web/#commentsFri, 29 Nov 2013 14:31:15 +0000http://chrislord.net/?p=233Continue reading →]]>Drawing on some of my limited HTML5 games experience, and marginally less limited general games and app writing experience, I’d like to write a bit about efficient animation for games on the web. I usually prefer to write about my experiences, rather than just straight advice-giving, so I apologise profusely for how condescending this will likely sound. I’ll try to improve in the future

There are a few things worth knowing that will really help your game (or indeed app) run better and use less battery life, especially on low-end devices. I think it’s worth getting some of these things down, as there’s evidence to suggest (in popular and widely-used UI libraries, for example) that it isn’t necessarily common knowledge. I’d also love to know if I’m just being delightfully/frustratingly naive in my assumptions.

First off, let’s get the basic stuff out of the way.

Help the browser help you

If you’re using DOM for your UI, which I’d certainly recommend, you really ought to use CSS transitions and/or animations, rather than JavaScript-powered animations. Though JS animations can be easier to express at times, unless you have a great need to synchronise UI animation state with game animation state, you’re unlikely to be able to do a better job than the browser. The reason for this is that CSS transitions/animations are much higher level than JavaScript, and express a very specific intent. Because of this, the browser can make some assumptions that it can’t easily make when you’re manually tweaking values in JavaScript. To take a concrete example, if you start a CSS transition to move something from off-screen so that it’s fully visible on-screen, the browser knows that the related content will end up completely visible to the user and can pre-render that content. When you animate position with JavaScript, the browser can’t easily make that same assumption, and so you might end up causing it to draw only the newly-exposed region of content, which may introduce slow-down. There are signals at the beginning and end of animations that allow you to attach JS callbacks and form a rudimentary form of synchronisation (though there are no guarantees on how promptly these callbacks will happen).

Speaking of assumptions the browser can make, you want to avoid causing it to have to relayout during animations. In this vein, it’s worth trying to stick to animating only transform and opacity properties. Though some browsers make some effort for other properties to be fast, these are pretty much the only ones semi-guaranteed to be fast across all browsers. Something to be careful of is that overflow may end up causing relayouting, or other expensive calculations. If you’re setting a transform on something that would overlap its container’s bounds, you may want to set overflow: hidden on that container for the duration of the animation.

Use requestAnimationFrame

When you’re animating canvas content, or when your DOM animations absolutely must synchronise with canvas content animations, do make sure to use requestAnimationFrame. Assuming you’re running in an arbitrary browsing session, you can never really know how long the browser will take to draw a particular frame. requestAnimationFrame causes the browser to redraw and call your function before that frame gets to the screen. The downside of using this vs. setTimeout, is that your animations must be time-based instead of frame-based. i.e. you must keep track of time and set your animation properties based on elapsed time. requestAnimationFrame includes a time-stamp in its callback function prototype, which you most definitely should use (as opposed to using the Date object), as this will be the time the frame began rendering, and ought to make your animations look more fluid. You may have a callback that ends up looking something like this:

You’ll note that I set startTime to -1 at the beginning, when I could just as easily set the time using the Date object and avoid the extra code in the animation callback. I do this so that any setup or processes that happen between the start of the animation and the callback being processed don’t affect the start of the animation, and so that all the animations I start before the frame is processed are synchronised.

To save battery life, it’s best to only draw when there are things going on, so that would mean calling requestAnimationFrame (or your refresh function, which in turn calls that) in response to events happening in your game. Unfortunately, this makes it very easy to end up drawing things multiple times per frame. I would recommend keeping track of when requestAnimationFrame has been called and only having a single handler for it. As far as I know, there aren’t solid guarantees of what order things will be called in with requestAnimationFrame (though in my experience, it’s in the order in which they were requested), so this also helps cut out any ambiguity. An easy way to do this is to declare your own refresh function that sets a flag when it calls requestAnimationFrame. When the callback is executed, you can unset that flag so that calls to that function will request a new frame again, like this:

Following this pattern, or something similar, means that no matter how many times you call requestRedraw, your drawing function will only be called once per frame.

Remember, that when you do drawing in requestAnimationFrame (and in general), you may be blocking the browser from updating other things. Try to keep unnecessary work outside of your animation functions. For example, it may make sense for animation setup to happen in a timeout callback rather than a requestAnimationFrame callback, and likewise if you have a computationally heavy thing that will happen at the end of an animation. Though I think it’s certainly overkill for simple games, you may want to consider using Worker threads. It’s worth trying to batch similar operations, and to schedule them at a time when screen updates are unlikely to occur, or when such updates are of a more subtle nature. Modern console games, for example, tend to prioritise framerate during player movement and combat, but may prioritise image quality or physics detail when compromise to framerate and input response would be less noticeable.

Measure performance

One of the reasons I bring this topic up, is that there exist some popular animation-related libraries, or popular UI toolkits with animation functions, that still do things like using setTimeout to drive their animations, drive all their animations completely individually, or other similar things that aren’t conducive to maintaining a high frame-rate. One of the goals for my game Puzzowl is for it to be a solid 60fps on reasonable hardware (for the record, it’s almost there on Galaxy Nexus-class hardware) and playable on low-end (almost there on a Geeksphone Keon). I’d have liked to use as much third party software as possible, but most of what I tried was either too complicated for simple use-cases, or had performance issues on mobile.

How I came to this conclusion is more important than the conclusion itself, however. To begin with, my priority was to write the code quickly to iterate on gameplay (and I’d certainly recommend doing this). I assumed that my own, naive code was making the game slower than I’d like. To an extent, this was true, I found plenty to optimise in my own code, but it go to the point where I knew what I was doing ought to perform quite well, and I still wasn’t quite there. At this point, I turned to the Firefox JavaScript profiler, and this told me almost exactly what low-hanging-fruit was left to address to improve performance. As it turned out, I suffered from some of the things I’ve mentioned in this post; my animation code had some corner cases where they could cause redraws to happen several times per frame, some of my animations caused Firefox to need to redraw everything (they were fine in other browsers, as it happens – that particular issue is now fixed), and some of the third party code I was using was poorly optimised.

A take-away

To help combat poor animation performance, I wrote Animator.js. It’s a simple animation library, and I’d like to think it’s efficient and easy to use. It’s heavily influenced by various parts of Clutter, but I’ve tried to avoid scope-creep. It does one thing, and it does it well (or adequately, at least). Animator.js is a fire-and-forget style animation library, designed to be used with games, or other situations where you need many, synchronised, custom animations. It includes a handful of built-in tweening functions, the facility to add your own, and helper functions for animating object properties. I use it to drive all the drawing updates and transitions in Puzzowl, by overriding its requestAnimationFrame function with a custom version that makes the request, but appends the game’s drawing function onto the end of the callback, like so:

My game’s redraw function does all drawing, and my animation callbacks just update state. When I request a redraw outside of animations, I just check the animator’s activeAnimations property first to stop from mistakenly drawing multiple times in a single animation frame. This gives me nice, synchronised animations at very low cost. Puzzowl isn’t out yet, but there’s a little screencast of it running on a Nexus 5:

]]>http://chrislord.net/index.php/2013/11/29/efficient-animation-for-games-on-the-mobile-web/feed/4Sabbatical Overhttp://chrislord.net/index.php/2013/10/28/sabbatical-over/
http://chrislord.net/index.php/2013/10/28/sabbatical-over/#commentsMon, 28 Oct 2013 09:22:11 +0000http://chrislord.net/?p=227Continue reading →]]>Aww, my 8-week sabbatical is now over. I wish I had more time, but I feel I used it well and there are certainly lots of Firefox bugs I want to work on too, so perhaps it’s about that time now (also, it’s not that long till Christmas anyway!)

So, what did I do on my sabbatical?

As I mentioned in the previous post, I took the time off primarily to work on a game, and that’s pretty much what I did. Except, I ended up working on two games. After realising the scope for our first game was much larger than we’d reckoned for, we decided to work on a smaller puzzle game too. I had a prototype working in a day, then that same prototype rewritten because DOM is slow in another day, then it rewritten again in another day because it ends up, canvas isn’t particularly fast either. After that, it’s been polish and refinement; it still isn’t done, but it’s fun to play and there’s promise. We’re not sure what the long-term plan is for this, but I’d like to package it with a runtime and distribute it on the major mobile app-stores (it runs in every modern browser, IE included).

The first project ended up being a first-person, rogue-like, dungeon crawler. None of those genres are known for being particularly brief or trivial games, so I’m not sure what we expected, but yes, it’s a lot of work. In this time, we’ve gotten our idea of the game a bit more solid, designed some interaction, worked on various bits of art (texture-sets, rough monsters) and have an engine that lets you walk around an area, pick things up and features deferred, per-pixel lighting. It doesn’t run very well on your average phone at the moment, and it has layout bugs in WebKit/Blink based browsers. IE11’s WebGL also isn’t complete enough to render it as it is, though I expect I could get a basic version of it working there. I’ve put this on the back-burner slightly to focus on smaller projects that can be demoed and completed in a reasonable time-frame, but I hope to have the time to return to it intermittently and gradually bring it up to the point where it’s recognisable as a game.

You can read a short paragraph and see a screenshot of both of these games at our team website, or see a few more on our Twitter feed.

What did I learn on my sabbatical?

Well, despite what many people are pretty eager to say, the web really isn’t ready as a games platform. Or an app platform, in my humble opinion. You can get around the issues if you have a decent knowledge of how rendering engines are implemented and a reasonable grasp of debugging and profiling tools, but there are too many performance and layout bugs for it to be comfortable right now, considering the alternatives. While it isn’t ready, I can say that it’s going to be amazing when it is. You really can write an app that, with relatively little effort, will run everywhere. Between CSS media queries, viewport units and flexbox, you can finally, easily write a responsive layout that can be markedly different for desktop, tablet and phone, and CSS transitions and a little JavaScript give you great expressive power for UI animations. WebGL is good enough for writing most mobile games you see, if you can avoid jank caused by garbage collection and reflow. Technologies like CocoonJS makes this really easy to deploy too.

Given how positive that all sounds, why isn’t it ready? These are the top bugs I encountered while working on some games (from a mobile specific viewpoint):

WebGL cannot be relied upon

WebGL has finally hit Chrome for Android release version, and has been enabled in Firefox and Opera for Android for ages now. The aforementioned CocoonJS lets you use it on iOS too, even. Availability isn’t the problem. The problem is that it frequently crashes the browser, or you frequently lose context, for no good reason. Changing the orientation of your phone, or resizing the browser on desktop has often caused the browser to crash in my testing. I’ve had lost contexts when my app is the only page running, no DOM manipulation is happening, no textures are being created or destroyed and the phone isn’t visibly busy with anything else. You can handle it, but having to recreate everything when this happens is not a great user experience. This happens frequently enough to be noticeable, and annoying. This seems to vary a lot per phone, but is not something I’ve experienced with native development at this scale.

An aside, Chrome also has an odd bug that causes a security exception if you load an image (on the same domain), render it scaled into a canvas, then try to upload that canvas. This, unfortunately, means we can’t use WebGL on Chrome in our puzzle game.

Canvas performance isn’t great

Canvas ought to be enough for simple 2d games, and there are certainly lots of compelling demos about, but I find it’s near impossible to get 60fps, full-screen, full-resolution performance out of even quite simple cases, across browsers. Chrome has great canvas acceleration and Firefox has an accelerated canvas too (possibly Aurora+ only at the moment), and it does work, but not well enough that you can rely on it. My puzzle game uses canvas as a fallback renderer on mobile, when WebGL isn’t an option, but it has markedly worse performance.

Porting to Chrome is a pain

A bit controversial, and perhaps a pot/kettle situation coming from a Firefox developer, but it seems that if Chrome isn’t your primary target, you’re going to have fun porting to it later. I don’t want to get into specifics, but I’ve found that Chrome often lays out differently (and incorrectly, according to specification) when compared to Firefox and IE10+, especially when flexbox becomes involved. Its transform implementation is also quite buggy too, and often ignores set perspective. There’s also the small annoyance that some features that are unprefixed in other browsers are still prefixed in Chrome (animations, 3d transforms). I actually found Chrome to be more of a pain than IE. In modern IE (10+), things tend to either work, or not work. I had fewer situations where something purported to work, but was buggy or incorrectly implemented.

Another aside, touch input in Chrome for Android has unacceptable latency and there doesn’t seem to be any way of working around it. No such issue in Firefox.

Appcache is awful

Uh, seriously. Who thought it was a good idea that appcache should work entirely independently of the browser cache? Because it isn’t a good idea. Took me a while to figure out that I have to change my server settings so that the browser won’t cache images/documents independently of appcache, breaking appcache updates. I tend to think that the most obvious and useful way for something to work should be how it works by default, and this is really not the case here.

Aside, Firefox has a bug that means that any two pages that have the same appcache manifest will cause a browser crash when accessing the second page. This includes an installed version of an online page using the same manifest.

CSS transitions/animations leak implementation details

This is the most annoying one, and I’ll make sure to file bugs about this in Firefox at least. Because setting of style properties gets coalesced, animations often don’t run. Removing display:none from an element and setting a style class to run a transition on it won’t work unless you force a reflow in-between. Similarly, switching to one style class, then back again won’t cause the animation on the first style-class to re-run. This is the case at least in Firefox and Chrome, I’ve not tested in IE. I can’t believe that this behaviour is explicitly specified, and it’s certainly extremely unintuitive. There are plenty of articles that talk about working around this, I’m kind of amazed that we haven’t fixed this yet. I’m equally concerned about the bad habits that this encourages too.

DOM rendering is slow

One of the big strengths of HTML5 as an app platform is how expressive HTML/CSS are and how you can easily create user interfaces in it, visually tweak and debugging them. You would naturally want to use this in any app or game that you were developing for the web primarily. Except, at least for games, if you use the DOM for your UI, you are going to spend an awful lot of time profiling, tweaking and making seemingly irrelevant changes to your CSS to try and improve rendering speed. This is no good at all, in my opinion, as this is the big advantage that the web has over native development. If you’re using WebGL only, you may as well just develop a native app and port it to wherever you want it, because using WebGL doesn’t make cross-device testing any easier and it certainly introduces a performance penalty. On the other hand, if you have a simple game, or a UI-heavy game, the web makes that much easier to work on. The one exception to this seems to be IE, which has absolutely stellar rendering performance. Well done IE.

This has been my experience with making web apps. Although those problems exist, when things come together, the result is quite beautiful. My puzzle game, though there are still browser-specific bugs to work around and performance issues to fix, works across varying size and specification of phone, in every major, modern browser. It even allows you to install it in Firefox as a dedicated app, or add it to your homescreen in iOS and Chrome beta. Being able to point someone to a URL to play a game, with no further requirement, and no limitation of distribution or questionable agreements to adheer to is a real game-changer. I love that the web fosters creativity and empowers the individual, despite the best efforts of various powers that be. We have work to do, but the future’s bright.

]]>http://chrislord.net/index.php/2013/10/28/sabbatical-over/feed/17Sabbaticalhttp://chrislord.net/index.php/2013/09/01/sabbatical/
http://chrislord.net/index.php/2013/09/01/sabbatical/#commentsSun, 01 Sep 2013 11:50:16 +0000http://chrislord.net/?p=221Continue reading →]]>As of Friday night, I am now on a two month unpaid leave. There are a few reasons I want to do this. It’s getting towards the 3-year point at Mozilla, and that’s usually the sort of time I get itchy feet to try something new. I also think I may have been getting a bit close to burn-out, which is obviously no good. I love my job at Mozilla and I think they’ve spoiled me too much for me to easily work elsewhere even if that wasn’t the case, so that’s one reason to take an extended break.

I still think Mozilla is a great place to work, where there are always opportunities to learn, to expand your horizons and to meet new people. An unfortunate consequence of that, though, is that I think it’s also quite high-stress. Not the kind of obvious stress you get from tight deadlines and other external pressures, but a more subtle, internal stress that you get from constantly striving to keep up and be the best you can be. Mozilla’s big enough now that it’s not uncommon to see people leave, but it does seem that a disproportionate amount of them cite stress or needing time to deal with life issues as part of the reason for moving on. Maybe we need to get better at recognising that, or at encouraging people to take more personal time?

Another reason though, and the primary reason, is that I want to spend some serious time working on creating a game. Those who know me know that I’m quite an avid gamer, and I’ve always had an interest in games development (I even spoke about it at Guadec some years back). Pre-employment, a lot of my spare time was spent developing games. Mostly embarrassingly poor efforts when I view them now, but it’s something I used to be quite passionate about. At some point, I think I decided that I preferred app development to games development, and went down that route. Given that I haven’t really been doing app development since joining Mozilla, it feels like a good time to revisit games development. If you’re interested in hearing about that, you may want to follow this Twitter account. We’ve started already, and I like to think that what we have planned, though very highly influenced by existing games, provides some fun, original twists. Let’s see how this goes

]]>http://chrislord.net/index.php/2013/09/01/sabbatical/feed/0Getting healthyhttp://chrislord.net/index.php/2013/07/13/getting-healthy/
http://chrislord.net/index.php/2013/07/13/getting-healthy/#commentsSat, 13 Jul 2013 08:58:36 +0000http://chrislord.net/?p=212Continue reading →]]>I’ve never really considered myself an unhealthy person. I exercise quite regularly and keep up with a reasonable amount of active hobbies (climbing, squash, tennis). That’s not really lapsed much, except for the time the London Mozilla office wasn’t ready and I worked at home – I think I climbed less during that period. Apparently though, that isn’t enough… After EdgeConf, I noticed in the recording of the session I participated in that I was looking a bit more plump than the mental image I had of myself. I weighed myself, and came to the shocking realisation that I was almost 14 stone (89kg). This put me well into the ‘overweight’ category, and was at least a stone heavier than I thought I was.

I’d long been considering changing my diet. I found Paul Rouget’s post particularly inspiring, and discussing diet with colleagues at various work-weeks had put ideas in my head. You could say that I was somewhat of a diet sceptic; I’d always thought that exercise was the key to maintaining a particular weight, especially cardiovascular exercise, and that with an active lifestyle you could get away with eating what you like. I’ve discovered that, for the most part, this was just plain wrong.

Before I go into the details of what I’ve done over the past 5 months, let me present some data:

I started my new diet on February 10th, and as of today (July 13th), I’ve lost 3 stone/~19kg and am well in the ‘ideal’ weight range. Of course, BMI is a pretty rough measure of anything, but I feel better, I’m in much better shape and I find physical activity far more enjoyable than I used to. So how did I do it?

One of the things that really intrigued me about Paul’s diet was that he said that all he did to lose the weight was change what he ate. Nothing else. This was a pretty enticing idea. I never thought I’d be able to give up things like pasta and bread, but if it really meant your weight would just start decreasing with no further effort, it almost seems silly not to give it a try… So I gave it a try. I cut out the major sources of carbohydrate in my diet (potato, pasta, rice, bread, snacks) and indeed my weight, as you can see, immediately started dropping. If you have the weight to lose, the results are pretty dramatic, and much faster than you’d expect. At this point, I was doing no extra exercise, and although I was snacking much less, my portion sizes for meals were unchanged.

I found some nice alternatives for the things I missed. Pasta and rice are quite nicely replaced by steamed, crushed cauliflower. Steamed aubergine makes a nice filler too. For potato, sweet potato is pretty much just better as far as I’m concerned, and celeriac is also a nice alternative. I never really found an alternative to bread, so I still have breakfast with my parents on the weekends and eat my mum’s home-made wholemeal bread. In moderation, I’ve not found it to interrupt my weight-loss at all. I still have porridge for breakfast, and I’m not strict about keeping to any particular amount of carbs in a day. If I gain weight on a day, I just try to be a bit more careful the next day.

The first two stone just dropped off. I did no extra exercise, I didn’t count my calories. The only thing I did was avoid high-carbohydrate foods and weight myself every day. There seem to be mixed opinions on weigh-in frequency, but being able to see the numbers go up and down was pretty significant motivation for me. Your mileage, as ever, may vary. After getting to about 12 stone, my fellow London Mozillian Jonathan Watt challenged me to beat him to 70kg. I’m very grateful for that, as at that time I was pretty happy with 12 stone (it’s in the ideal range and it felt and looked noticeable to me).

The next stone, though still reasonably easy, didn’t come without effort. However, this increased effort was enjoyable, and is now a part of my life (and I intend that to continue). It ends up that carrying 12kg less weight while climbing makes it much more enjoyable, so I was able to climb longer and more frequently. Similarly, I started running with another colleague, Ryan Watson. The weight continued to come off, if anything at an increased rate now, and I was reaching weights I hadn’t been since my early teens.

The last few pounds has been difficult though. I wanted to hit 10 stone 12 to say that I lost 3 stone exactly (perhaps a slightly obsessive compulsion with whole numbers), but realistically, I think 70kg/~11 stone is the weight I’ll maintain. I’m now training for strength to change my body composition to something that will more easily allow me to maintain this weight.

A lot of people helped me to get this far. Ryan was especially encouraging and helped me train when I started to up the exercise. Without Jonathan’s competition, I may have settled for a weight that was still really above the weight I should be and last but not least, I have to thank my wonderful partner Laura for accommodating my new diet and helping me find lots of tasty things to eat. Not to mention my wonderful friends, family and colleagues, all of whom have been terrifically encouraging and supportive. Thanks, everyone

]]>http://chrislord.net/index.php/2013/07/13/getting-healthy/feed/12Writing and deploying a small Firefox OS applicationhttp://chrislord.net/index.php/2013/05/04/writing-and-deploying-a-small-firefox-os-application/
http://chrislord.net/index.php/2013/05/04/writing-and-deploying-a-small-firefox-os-application/#commentsSat, 04 May 2013 08:37:53 +0000http://chrislord.net/?p=202Continue reading →]]>For the last week I’ve been using a Geeksphone Keon as my only phone. There’s been no cheating here, I don’t have a backup Android phone and I’ve not taken to carrying around a tablet everywhere I go (though its use has increased at home slightly…) On the whole, the experience has been positive. Considering how entrenched I was in Android applications and Google services, it’s been surprisingly easy to make the switch. I would recommend anyone getting the Geeksphones to build their own OS images though, the shipped images are pretty poor.

Among the many things I missed (Spotify is number 1 in that list btw), I could have done with a countdown timer. Contrary to what the interfaces of most Android timer apps would have you believe, it’s not rocket-science to write a usable timer, so I figured this would be a decent entry-point into writing mobile web applications. For the most part, this would just be your average web-page, but I did want it to feel ‘native’, so I started looking at the new building blocks site that documents the FirefoxOS shared resources. I had elaborate plans for tabs and headers and such, but turns out, all I really needed was the button style. The site doesn’t make hugely clear that you’ll actually need to check out the shared resources yourself, which can be found on GitHub.

Writing the app was easy, except perhaps for getting things to align vertically (for which I used the nested div/”display: table-cell; vertical-alignment: middle;” trick), but it was a bit harder when I wanted to use some of the new APIs. In particular, I wanted the timer to continue to work when the app is closed, and I wanted it to alert you only when you aren’t looking at it. This required use of the Alarm API, the Notifications API and the Page Visibility API.

The page visibility API was pretty self-explanatory, and I had no issues using it. I use this to know when the app is put into the background (which, handily, always happens before closing it. I think). When the page gets hidden, I use the Alarm API to set an alarm for when the current timer is due to elapse to wake up the application. I found this particularly hard to use as the documentation is very poor (though it turns out the code you need is quite short). Finally, I use the Notifications API to spawn a notification if the app isn’t visible when the timer elapses. Notifications were reasonably easy to use, but I’ve yet to figure out how to map clicking on a notification to raising my application – I don’t really know what I’m doing wrong here, any help is appreciated!Update: Thanks to Thanos Lefteris in the comments below, this now works – activating the notification will bring you back to the app.

The last hurdle was deploying to an actual device, as opposed to the simulator. Apparently the simulator has a deploy-to-device feature, but this wasn’t appearing for me and it would mean having to fire up my Linux VM (I have my reasons) anyway, as there are currently no Windows drivers for the Geeksphone devices available. I obviously don’t want to submit this to the Firefox marketplace yet, as I’ve barely tested it. I have my own VPS, so ideally I could just upload the app to a directory, add a meta tag in the header and try it out on the device, but unfortunately it isn’t as easy as that.

Getting it to work well as a web-page is a good first step, and to do that you’ll want to add a meta viewport tag. Getting the app to install itself from that page was easy to do, but difficult to find out about. I think the process for this is harder than it needs to be and quite poorly documented, but basically, you want this in your app:

And you want all paths in your manifest and appcache manifest to be absolute (you can assume the host, but you can’t have paths relative to the directory the files are in). This last part makes deployment very awkward, assuming you don’t want to have all of your app assets in the root directory of your server and you don’t want to setup vhosts for every app. You also need to make sure your server has the webapp mimetype setup. Mozilla has a great online app validation tool that can help you debug problems in this process.

Visiting the page will offer to install the app for you on a device that supports app installation (i.e. a Firefox OS device). Not bad for a night’s work! Feel free to laugh at my n00b source and tell me how terrible it is in the comments

]]>http://chrislord.net/index.php/2013/05/04/writing-and-deploying-a-small-firefox-os-application/feed/11Tips for smooth scrolling web pages (EdgeConf follow-up)http://chrislord.net/index.php/2013/02/11/tips-for-smooth-scrolling-web-pages-edgeconf-follow-up/
http://chrislord.net/index.php/2013/02/11/tips-for-smooth-scrolling-web-pages-edgeconf-follow-up/#commentsMon, 11 Feb 2013 10:07:46 +0000http://chrislord.net/?p=178Continue reading →]]>I’m starting to type this up as EdgeConf draws to a close. I spoke on the performance panel, with Shane O’Sullivan, Rowan Beentje and Pavel Feldman, moderated by Matt Delaney, and tried to bring a platform perspective to the affair. I found the panel very interesting, and it reminded me how little I know about the high-level of web development. Similarly, I think it also highlighted how little consideration there usually is for the platform when developing for the web. On the whole, I think that’s a good thing (platform details shouldn’t be important, and they have a habit of changing), but a little platform knowledge can help in structuring things in a way that will be fast today, and as long as it isn’t too much of a departure from your design, it doesn’t hurt to think about it. At one point in the panel, I listed a few things that are particularly slow from a platform perspective today. While none of these were intractable problems, they may not be fixed in the near future and feedback indicated that they aren’t all common knowledge. So what follows are a few things to avoid, and a few things to do that will help make your pages scroll smoothly on both desktop and mobile. I’m going to try not to write lies, but I hope if I get anything slightly or totally wrong, that people will correct me in the comments and I can update the post accordingly

Avoid reflow

When I mentioned this at the conference, I prefaced it with a quick explanation of how rendering a web page works. It’s probably worth reiterating this. After network and such have happened and the DOM tree has been created, this tree gets translated into what we call the frame tree. This tree is similar to the DOM tree, but it’s structured in a way that better represents how the page will be drawn. This tree is then iterated over and the size and position of these frames are calculated. The act of calculating these positions and sizes is referred to as reflow. Once reflow is done, we translate the frame tree into a display list (other engines may skip this step, but it’s unimportant), then we draw the display list into layers. Where possible, we keep layers around and only redraw parts that have changed/newly become visible.

Really, reflow is actually quite fast, or at least it can be, but it often forces things to be redrawn (and drawing is often slow). Reflow happens when the size or position of things changes in such a way that dependent positions and sizes of elements need to be recalculated. Reflow usually isn’t something that will happen to the entire page at once, but incautious structuring of the page can result in this. There are quite a few things you can do to help avoid large reflows; set widths and heights to absolute values where possible, don’t reposition or resize things, don’t unnecessarily change the style of things. Obviously these things can’t always be avoided, but it’s worth thinking if there are other ways to achieve the result you want that don’t force reflow. If positions of things must be changed, consider using a CSS translate transform, for example – transforms don’t usually cause reflow.

If you absolutely have to do something that will trigger reflow, it’s important to be careful how you access properties in JavaScript. Reflow will be delayed as long as possible, so that if multiple things happen in quick succession that would cause reflow, only a single reflow actually needs to happen. If you access a property that relies on the frame tree being up to date though, this will force reflow. Practically, it’s worth trying to batch DOM changes and style changes, and to make sure that any property reads happen outside of these blocks. Interleaving reads and writes can end up forcing multiple reflows per page-draw, and the cost of reflow can add up quickly.

Avoid drawing

This sounds silly, but you should really only make the browser do as little drawing as is absolutely necessary. Most of the time, drawing will happen on reflow, when new content appears on the screen and when style changes. Some practical advice to avoid this would be to avoid making DOM changes near the root of the tree, avoid changing the size of things and avoid changing text (text drawing is especially slow). While repositioning doesn’t always force redrawing, you can ensure this by repositioning using CSS translate transforms instead of top/left/bottom/right style properties. Especially avoid causing redraws to happen while the user is scrolling. Browsers try their hardest to keep up the refresh rate while scrolling, but there are limits on memory bandwidth (especially on mobile), so every little helps.

Thinking of things that are slow to draw, radial gradients are very slow. This is really just a bug that we should fix, but if you must use CSS radial gradients, try not to change them, or put them in the background of elements that frequently change.

Avoid unnecessary layers

One of the reasons scrolling can be fast at all on mobile is that we reduce the page to a series of layers, and we keep redrawing on these layers down to a minimum. When we need to redraw the page, we just paste these layers that have already been drawn. While the GPU is pretty great at this, there are limits. Specifically, there is a limit to the amount of pixels that can be drawn on the screen in a certain time (fill-rate) – when you draw to the same pixel multiple times, this is called overdraw, and counts towards the fill-rate. Having lots of overlapping layers often causes lots of overdraw, and can cause a frame’s maximum fill-rate to be exceeded.

This is all well and good, but how does one avoid layers at a high level? It’s worth being vaguely aware of what causes stacking contexts to be created. While layers usually don’t exactly correspond to stacking contexts, trying to reduce stacking contexts will often end up reducing the number of resulting layers, and is a reasonable exercise. Even simpler, anything with position: fixed, background-attachment: fixed or any kind of CSS transformed element will likely end up with its own layer, and anything with its own layer will likely force a layer for anything below it and anything above it. So if it’s not necessary, avoid those if possible.

What can we do at the platform level to mitigate this? Firefox already culls areas of a layer that are made inaccessible by occluding layers (at least to some extent), but this won’t work if any of the layers end up with transforms, or aren’t opaque. We could be smarter with culling for opaque, transformed layers, and we could likely do a better job of determining when a layer is opaque. I’m pretty sure we could be smarter about the culling we already do too.

Avoid blending

Another thing that slows down drawing is blending. This is when the visual result of an operation relies on what’s already there. This requires the GPU (or CPU) to read what’s already there and perform a calculation on the result, which is of course slower than just writing directly to the buffer. Blending also doesn’t interact well with deferred rendering GPUs, which are popular on mobile.

This alone isn’t so bad, but combining it with text rendering is not so great. If you have text that isn’t on a static, opaque background, that text will be rendered twice (on desktop at least). First we render it on white, then on black, and we use those two buffers to maintain sub-pixel anti-aliasing as the background varies. This is much slower than normal, and also uses up more memory. On mobile, we store opaque layers in 16-bit colour, but translucent layers are stored in 32-bit colour, doubling the memory requirement of a non-opaque layer.

We could be smarter about this. At the very least, we could use multi-texturing and store non-opaque layers as a 16-bit colour + 8-bit alpha, cutting the memory requirement by a quarter and likely making it faster to draw. Even then, this will still be more expensive than just drawing an opaque layer, so when possible, make sure any text is on top of a static, opaque background when possible.

Avoid overflow scrolling

The way we make scrolling fast on mobile, and I believe the way it’s fast in other browsers too, is that we render a much larger area than is visible on the screen and we do that asynchronously to the user scrolling. This works as the relationship between time and size of drawing is not linear (on the whole, the more you draw, the cheaper it is per pixel). We only do this for the content document, however (not strictly true, I think there are situations where whole-page scrollable elements that aren’t the body can take advantage of this, but it’s best not to rely on that). This means that any element that isn’t the body that is scrollable can’t take advantage of this, and will redraw synchronously with scrolling. For small, simple elements, this doesn’t tend to be a problem, but if your entire page is in an iframe that covers most or all of the viewport, scrolling performance will likely suffer.

On desktop, currently, drawing is synchronous and we don’t buffer area around the page like on mobile, so this advice doesn’t apply there. But on mobile, do your best to avoid using iframes or having elements that have overflow that aren’t the body. If you’re using overflow to achieve a two-panel layout, or something like this, consider using position:fixed and margins instead. If both panels must scroll, consider making the largest panel the body and using overflow scrolling in the smaller one.

I hope we’ll do something clever to fix this sometime, it’s been at the back of my mind for quite a while, but I don’t think scrolling on sub-elements of the page can ever really be as good as the body without considerable memory cost.

Take advantage of the platform

This post sounds all doom and gloom, but I’m purposefully highlighting what we aren’t yet good at. There are a lot of things we are good at (or reasonable, at least), and having a fast page need not necessarily be viewed as lots of things to avoid, so much as lots of things to do.

Although computing power continues to increase, the trend now is to bolt on more cores and more hardware threads, and the speed increase of individual cores tends to be more modest. This affects how we improve performance at the application level. Performance increases, more often than not, are about being smarter about when we do work, and to do things concurrently, more than just finding faster algorithms and micro-optimisation.

This relates to the asynchronous scrolling mentioned above, where we do the same amount of work, but at a more opportune time, and in a way that better takes advantage of the resources available. There are other optimisations that are similar with regards to video decoding/drawing, CSS animations/transitions and WebGL buffer swapping. A frequently occurring question at EdgeConf was whether it would be sensible to add ‘hints’, or expose more internals to web developers so that they can instrument pages to provide the best performance. On the whole, hints are a bad idea, as they expose platform details that are liable to change or be obsoleted, but I think a lot of control is already given by current standards.

On a practical level, take advantage of CSS animations and transitions instead of doing JavaScript property animation, take advantage of requestAnimationFrame instead of setTimeout, and if you find you need even more control, why not drop down to raw GL via WebGL, or use Canvas?

I hope some of this is useful to someone. I’ll try to write similar posts if I find out more, or there are significant platform changes in the future. I deliberately haven’t mentioned profiling tools, as there are people far more qualified to write about them than I am. That said, there’s a wiki page about the built-in Firefox profiler, some nice documentation on Opera’s debugging tools and Chrome’s tools look really great too.

]]>http://chrislord.net/index.php/2013/02/11/tips-for-smooth-scrolling-web-pages-edgeconf-follow-up/feed/15Firefox for Android in 2013http://chrislord.net/index.php/2013/02/07/firefox-for-android-in-2013/
http://chrislord.net/index.php/2013/02/07/firefox-for-android-in-2013/#commentsThu, 07 Feb 2013 15:31:06 +0000http://chrislord.net/?p=173Continue reading →]]>Lucas Rocha and I gave a talk at FOSDEM over the weekend on Firefox for Android. It went ok, I think we could have rehearsed it a bit better, but it was generally well-received and surprisingly well-attended! I’m sure Lucas will have the slides up soon too. If you were unfortunate enough not to have attended FOSDEM, and doubly unfortunate that you missed our talk (guffaw), we’ll be reiterating it with a bit more detail in the London Mozilla space on February 22nd. We’ll do our best to answer any questions you have about Firefox for Android, but also anything Mozilla-related. If you’re interested in FirefoxOS, there may be a couple of phones knocking about too. Do come along, we’re looking forward to seeing you

p.s. I’ll be talking on a performance panel at EdgeConf this Saturday. Though it’s fully booked, I think tickets occasionally become available again, so might be worth keeping an eye on. They’ll be much cleverer people than me knocking about, but I’ll be doing my best to answer your platform performance related questions.

]]>http://chrislord.net/index.php/2013/02/07/firefox-for-android-in-2013/feed/3Progressive Tile Renderinghttp://chrislord.net/index.php/2012/10/17/progressive-tile-rendering/
http://chrislord.net/index.php/2012/10/17/progressive-tile-rendering/#commentsWed, 17 Oct 2012 15:39:00 +0000http://Software/Mozilla/progressive-tile-renderingContinue reading →]]>So back from layout into graphics again! For the last few weeks, I’ve been working with Benoit Girard on getting progressive tile rendering finished and turned on by default in Firefox for Android. The results so far are very promising! First, a bit of background (feel free to skip to the end if you just want the results).

You may be aware that we use a multi-threaded application model for Firefox for Android. The UI runs in one thread and Gecko, which does the downloading and rendering of the page, runs in another. This is a bit of a simplification, but for all intents and purposes, that’s how it works. We do this so that we can maintain interactive performance – something of paramount important with a touch-screen. We render a larger area than you see on the screen, so that when you scroll, we can respond immediately without having to wait for Gecko to render more. We try to tell Gecko to render the most relevant area next and we hope that it returns in time so that the appearance is seamless.

There are two problems with this as it stands, though. If the work takes too long, you’ll be staring at a blank area (well, this isn’t quite true either, we do a low-resolution render of the entire page and use that as a backing in this worst-case scenario – but that often doesn’t work quite right and is a performance issue in and of itself…) The second problem is that if a page is made up of many layers, or updates large parts of itself as you scroll, uploading that work to the graphics unit can take a significant amount of time. During this time, the page will appear to ‘hang’, as unfortunately, you can’t upload data to the GPU and continue to use it to draw things (this isn’t true in every single case, but again, for our purposes, it is).

Progressive rendering tries to spread this load by breaking up that work into several smaller tiles, and processing them one-by-one, where appropriate. This helps us mitigate those pauses that may happen for particularly complex/animated pages. Alongside this work, we also add the ability for a render to be cancelled. This is good for the situation that a page takes so long to render that by the time it’s finished, what it rendered is no longer useful. Currently, because a render is done all at once, if it takes too long, we can waste precious cycles on irrelevant data. As well as splitting up this work, and allowing it to be cancelled, we also try to do it in the most intelligent order – render areas that the user can see that were previously blank first, and if that area intersects with more than one tile, make sure to do it in the order that maintains visual coherence the best.

A cherry on the top (which is still very much work-in-progress, but I hope to complete it soon), is that splitting this work up into tiles makes it easy to apply nice transitions to make the pathological cases not look so bad. With that said, how’s about some video evidence? Here’s an almost-Nightly (an extra patch or two that haven’t quite hit central), with the screenshot layer disabled so you can see what can happen in a pathological case:

And here’s the same code, with progressive tile rendering turned on and a work-in-progress fading patch applied.

This page is a particularly slow page to render due to the large radial gradient in the background (another issue which will eventually be fixed), so it helps to highlight how this work can help. For a fast-to-render page that we have no problems with, this work doesn’t have such an obvious effect (though scrolling will still be smoother). I hope the results speak for themselves