Jekyll2019-02-11T08:32:59+00:00https://nooshu.github.io/feed.xmlFrontend Web Developer, Oxfordshire, UK - Matt HobbsThe virtual notebook of Matt Hobbs, about web development and any other subject that interests him. UI / Frontend Developer.
Matt HobbsJoining the 100 club (#webperf)2018-12-06T12:09:00+00:002018-12-06T12:09:00+00:00https://nooshu.github.io/blog/2018/12/06/joining-the-100-club<p>So a few months back we made a couple of changes to GOV.UK to <a href="https://gdstechnology.blog.gov.uk/2018/10/04/making-gov-uk-pages-load-faster-and-use-less-data/">improve frontend performance</a> for all users. I’d been pushing to make this change for a number of months, but unfortunately changes in government tend to take quite a while. So although it was a fairly simple change, the impact it had on other GDS projects was noticeable. This increased the implementation timeline quite considerably. Since this change was made we’ve been looking to improve other areas of the website to make it even faster. In doing so I’ve been doing a lot of reading and experimenting with web performance. That got me thinking “I wonder how well my website performs?”.</p>
<h2 id="initial-results">Initial results</h2>
<p>I used an excellent tool called <a href="https://developers.google.com/web/tools/lighthouse/">Google Lighthouse</a> that is built into Chrome to check the overall performance of the site. The initial results were good, but not perfect. The tests were conducted on the default Lighthouse settings for mobile with “Applied Fast 3G, 4x CPU Slowdown” selected. You can see the <a href="https://googlechrome.github.io/lighthouse/viewer/?gist=e185d8f59f07e714f7f188a0f12b45e4">full results from the audit here</a>.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/lighthouse-before.jpg" alt="Lighthouse score for this website before optimisation." />
<figcaption class="figure__caption">Performance score 97/100. Good but not perfect.</figcaption>
</figure>
<p>As you can see the score was 97 out of 100. It’s a decent score, but where were the last 3 points being lost? Well there’s a handy Google sheet called <a href="https://docs.google.com/spreadsheets/d/1dXH-bXX3gxqqpD1f7rp6ImSOhobsT1gn_GQ2fGZp8UU/edit?ts=59fb61d2#gid=0">‘Lighthouse Score Weighting’</a> that you can examine. It tells you what you should be scoring for each performance metric to achieve a certain percentage:</p>
<ul>
<li>First Contentful Paint</li>
<li>First Meaningful Paint</li>
<li>Speed Index</li>
<li>First CPU Idle</li>
<li>Time to Interactive</li>
</ul>
<p>A comparison between the two sets of data showed there was a fair bit of improvement needed to hit 100. Time to look at the network tab and run an audit on <a href="https://www.webpagetest.org/">Web Page Test</a>.</p>
<h2 id="what-were-the-issues">What were the issues?</h2>
<p>From the audit run on Web Page Test it was easy to spot where some improvements could be made. The results shown were run on an iPhone 5c on a 4G connection. Not the most up-to-date handset, but still one that is very popular.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/waterfall-before.jpg" alt="Web Page Test waterfall graph before" />
<figcaption class="figure__caption">A waterfall graph is a great way to spot inefficiencies in network performance.</figcaption>
</figure>
<p>On the graph you can see some green, orange and purple bands. This is DNS lookup, connection established and SSL negotiation. Each of these is taking 800ms. Now that doesn’t sound like much, but in web performance terms that is huge! Each of these lookups was for a Google service: Analytics and Google Fonts. As I want to monitor the analytics data, there’s not much I can do about that. For the fonts there are certainly optimisations that can be made.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/filmstrip-before.jpg" alt="The WPT filmstrip shows where the issue is: font performance." />
<figcaption class="figure__caption">The filmstrip view shows exactly what the issue is: Font performance isn't great. iPhone 5C 10.3 (no preload support)</figcaption>
</figure>
<p>The filmstrip view from Web Page Test really highlights the main issue. Notice how 3 seconds into loading we have the background colours but no text. At this point the font hasn’t been downloaded and what we are seeing is a Flash of Invisible Text (FOIT). This will be killing the First Meaningful Paint (FMP) score which measures when a user perceives that the primary content of a page is visible. It’s easy to see why this is all happening when you combine the waterfall graph with the filmstrip. The fonts are being hosted on a 3rd party website, so there will be a separate set of connection negotiation and round trip time associated with the font download. Time to eliminate these issues. The full Web Page Test report can be <a href="https://www.webpagetest.org/result/181201_W7_a2e23b029f4b4b83963a43f0eeced8fe/">seen here</a>.</p>
<h2 id="what-did-i-change">What did I change?</h2>
<h3 id="eliminate-the-3rd-party-hosting-google-fonts">Eliminate the 3rd-party hosting (Google Fonts)</h3>
<p>Is it possible? Sure it is. Thankfully there is a really helpful tool to allow you to do this easily, <a href="https://google-webfonts-helper.herokuapp.com/fonts">google-webfonts-helper</a>. Select the font you want to self-host, download and away you go. Place the files on the same server as your main website so you are no longer dependant on a 3rd party for your fonts.</p>
<h3 id="remove-unneeded-fonts-swap-for-a-system-font">Remove unneeded fonts, swap for a system font</h3>
<p>Did I really need the second webfont for the main content? In using two Google Fonts, this is doubling up on the number of fonts to download and also the associated CSS. More bytes to download equals slower performance. So I swapped the main content font to <code class="highlighter-rouge">Geneva</code>, as it is very similar to what I had before. Now the browser can use a locally installed font rather than having to download the second font.</p>
<h3 id="control-the-way-fonts-are-displayed-while-loading">Control the way fonts are displayed while loading</h3>
<p>How can we control the way the font loads? Well there’s a CSS property for that <code class="highlighter-rouge">font-display</code>. At the moment browser support <a href="https://caniuse.com/#search=font-display">isn’t perfect</a>, but it is getting there. This setting allows a developer to control the <a href="https://font-display.glitch.me/">block, swap and failure period</a> when loading. I’ve chosen to use <code class="highlighter-rouge">font-display: swap;</code>. What this does is:</p>
<ul>
<li>There’s no block period, so before the custom font is loaded the fallback font is displayed</li>
<li>Once the font is downloaded, the fallback will be swapped for the custom font. There is an infinite period in which this can happen.</li>
<li>Downside to this setting, you could potentially see Flash of Unstyled Text (FOUT)</li>
</ul>
<p>So why don’t Google Fonts use this setting by default? Why can it only be applied when you self-host the fonts? Well there’s a <a href="https://github.com/google/fonts/issues/358">really long thread discussing just that</a> if you are interested.</p>
<h3 id="minimise-the-chance-of-fout">Minimise the chance of FOUT</h3>
<p>Is it possible to minimise the likelihood of the FOUT? Yes, and it is super simple to implement. What is needed is the <code class="highlighter-rouge">preload</code> link element in the <code class="highlighter-rouge">&lt;head&gt;</code> tag. This allows a developer to tell the browser about critical assets that will need to be downloaded in the future. By telling the browser ahead of time that the font will be required, it should therefore be downloaded by the time it is actually required on the page. If the font is already in the cache, no FOUT!</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;link</span> <span class="na">rel=</span><span class="s">"preload"</span> <span class="na">href=</span><span class="s">"/assets/font.woff2"</span> <span class="na">as=</span><span class="s">"font"</span> <span class="na">type=</span><span class="s">"font/woff2"</span><span class="nt">&gt;</span>
</code></pre></div></div>
<p>It is worth checking out the <a href="https://beta.caniuse.com/#search=preload">browser support</a> for this feature, as it is still quite new. And a word of warning about its usage: use it sparingly for assets you know the page absolutely needs. The preloaded asset will be bumped to the highest priority in the queue. It will be prioritised even higher than CSS. So if you preload a lot of files, you’ve just added a large number of render blocking bytes to the <a href="https://developers.google.com/web/fundamentals/performance/critical-rendering-path/">critical path</a>!</p>
<p>I’ve run a <a href="https://www.webpagetest.org/video/compare.php?tests=181205_X7_82bdf910748d7390556e3c9ca21f62cf%2C181205_4M_c596e12ea22aa279d6ee1fd26518094e&amp;thumbSize=200&amp;ival=100&amp;end=visual">couple of tests on WPT</a> and compared them if you’d like to see the difference <code class="highlighter-rouge">preload</code> can make on supporting browsers. iPhone 5 SE on a 4G connection visually completes 500ms quicker with <code class="highlighter-rouge">preload</code>!</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/preload-in-action.png" alt="Web Page Test waterfall with preloaded font visible." />
<figcaption class="figure__caption">Preload in action on the iPhone SE iOS 12. Notice the red bar directly after the initial HTML download.</figcaption>
</figure>
<p>The full specification for preload can be <a href="https://www.w3.org/TR/preload/">found here</a>, along with the section related to <a href="https://www.w3.org/TR/preload/#early-fetch-of-critical-resources">fetching critical assets</a>.</p>
<h2 id="final-results">Final results</h2>
<p>Well I’m happy to report that I managed to join the “100 club”! A completely made up club for people who’s website scores 100/100 for performance in Lighthouse. Silliness aside, the results were really impressive considering how little work it required. The full audit for the after performance can be <a href="https://googlechrome.github.io/lighthouse/viewer/?gist=e4a712467b36cd2964d627e82a078be4">viewed here</a>.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/lighthouse-after.jpg" alt="Lighthouse score for this website after optimisation." />
<figcaption class="figure__caption">Performance score 100/100. Huzzah!</figcaption>
</figure>
<p>The key metrics were all reduced by the following amounts:</p>
<ul>
<li>First Contentful Paint: 630ms</li>
<li>First Meaningful Paint: 1,780ms</li>
<li>Speed Index: 810ms</li>
<li>First CPU Idle: 460ms</li>
<li>Time to Interactive: 460ms</li>
</ul>
<p>That’s an incredible increase in performance, especially around the First Meaningful Paint (when the primary content is visible to the user). Almost 1.8 seconds quicker!</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/waterfall-after.jpg" alt="Web Page Test waterfall graph after" />
<figcaption class="figure__caption">Notice how the number of DNS/Connect/SSL bars has been reduced. No preload support on iPhone 5c 10.3 though :(</figcaption>
</figure>
<p>The changes can really be seen in the waterfall graph. Notice how there are only 2 DNS/Connect/SSL negotiations in this graph (there were 4 before the changes were made). The font is downloaded immediately since the initial connection negotiation to the main domain has already been established.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/filmstrip-after.jpg" alt="New improved filmstrip shows increased performance" />
<figcaption class="figure__caption">Notice how there are fewer panels than before and only a single panel with the font not visible. iPhone 5C 10.3 (no preload support)</figcaption>
</figure>
<p>Overall page load time and the number of panels without main content have been reduced. The perceived performance of the site has been greatly improved for users. The full Web Page Test report can be <a href="https://www.webpagetest.org/result/181201_48_b631ffc49a5c7f2b85e66a4806cd00f3/">seen here</a>.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/12/load-comparison.gif" alt="Animation showing the difference in page load performance." />
<figcaption class="figure__caption">A visualisation of what the changes have made to page load. iPhone 5C 10.3 (no preload support)</figcaption>
</figure>
<p>Web Page Test actually comes with a really handy feature. You can compare multiple results on the same page, allowing you to see exactly what has changed. Here’s a comparison of the <a href="https://www.webpagetest.org/video/compare.php?tests=181201_48_b631ffc49a5c7f2b85e66a4806cd00f3%2C181201_W7_a2e23b029f4b4b83963a43f0eeced8fe&amp;thumbSize=200&amp;ival=500&amp;end=doc">before and after audits</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>By using freely available performance tools like Lighthouse and Web Page Test, it is possible to easily identify performance bottleneck’s in your website’s frontend code. Solving those performance issues once identified can sometimes a more difficult task. Thankfully the optimisation of webfont performance has been a high priority for browser vendors over the past few years. We as developers have been given tools that can make a real difference in web performance. This should making the web faster for all users as they become more widely adopted.</p>Matt HobbsIt's amazing what a few font tweaks can do to perceived performance. Now lighthouse hits 100/100 for performance!Making GOV.UK pages load faster and use less data2018-10-04T12:09:00+00:002018-10-04T12:09:00+00:00https://nooshu.github.io/blog/2018/10/04/making-gov-uk-pages-load-faster-and-use-less-data<hr />
<p><strong>Note</strong>: This post was originally posted on the <a href="https://gdstechnology.blog.gov.uk/">GDS Technology blog</a>. It can be <a href="https://gdstechnology.blog.gov.uk/2018/10/04/making-gov-uk-pages-load-faster-and-use-less-data/">viewed here</a>.</p>
<hr />
<figure class="figure">
<img class="figure__image" src="/images/2018/08/gds-typography-1.jpg" alt="GDS testing the New Transport font weights." />
</figure>
<p>We’ve used a custom version of the <a href="https://gds.blog.gov.uk/2012/07/05/a-few-notes-on-typography/">New Transport font</a> since the launch of <a href="https://www.gov.uk/government/news/launch-of-gov-uk-a-key-milestone-in-making-public-service-delivery-digital-by-default">GOV.UK</a>. New Transport was rendered on-screen by <a href="https://en.wikipedia.org/wiki/Base64">Base64 encoding</a> font files into a CSS file. Base64 encoding was the preferred technique when we launched in 2012 as it helped to reduce the number of HTTP requests and minimise the <a href="https://en.wikipedia.org/wiki/Flash_of_unstyled_content">flash of unstyled content</a> (FOUC).</p>
<h2 id="why-we-changed-font-loading-on-govuk">Why we changed font loading on GOV.UK</h2>
<p>Browsers have improved a lot since 2012 and Base64 encoding font files is now <a href="https://csswizardry.com/2017/02/base64-encoding-and-performance/">considered a method you should avoid</a>. The main problems we had with the Base64 encoding method were that we:</p>
<ul>
<li>forced browsers to download fonts even if they were not used</li>
<li>could only use the <a href="https://developer.mozilla.org/en-US/docs/Web/Guide/WOFF">Web Open Font Format</a> (WOFF) because of the file size restriction</li>
</ul>
<p>Base64 encoding also added extra bytes to the page that were sitting on the browsers’ critical rendering path. All these bytes needed to be loaded before the page could be rendered. The CSS file was approximately 200 kilobytes (KB), which was 82% of the total blocking assets needed to load the page.</p>
<h2 id="how-we-changed-font-loading-on-govuk">How we changed font loading on GOV.UK</h2>
<p>We made a simple code change but the impact on the wider GOV.UK codebase was significant. We changed the way the <code class="highlighter-rouge">@font-face</code> loading method referenced fonts. By removing the inline Base64 encoded font and loading the files via standard HTTP requests we can now:</p>
<ul>
<li>serve multiple font types from the same <code class="highlighter-rouge">@font-face</code> declaration, which allows a browser to choose the appropriate font it supports</li>
<li>remove approximately 200KB from the critical path to load pages faster</li>
<li>start to take advantage of modern browser optimisation techniques like the <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/@font-face/font-display">CSS font-display property</a> and <a href="https://developer.mozilla.org/en-US/docs/Web/HTML/Preloading_content">resource preloading</a></li>
</ul>
<figure class="figure">
<img class="figure__image" src="/images/2018/08/gds-typography-2.png" alt="New Transport font sizes in the Design System." />
</figure>
<h2 id="the-main-advantages-for-end-users">The main advantages for end users</h2>
<h3 id="less-data-usage">Less data usage</h3>
<p>Removing the Base64 encoded font has reduced the total page weight by 16% (75 KB) per request (assuming no caching). This may not sound like a huge difference, but GOV.UK receives approximately 48 million visitors per month, so this adds up to mobile users saving approximately 800 GB per month, cumulatively. This is especially important to users on older mobile devices and expensive data plans.</p>
<p>By removing the Base64 encoding we are also giving mobile users the ability to opt in or out of downloading the font completely. This can help users reduce their data consumption.</p>
<h3 id="faster-page-loading">Faster page loading</h3>
<p>We have seen big improvements in page load time since we introduced the change. In our testing, the page load time for high-end devices using 4G dropped by 13% and visual population of the page (<a href="https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index">Speed Index</a>) reduced by 50%.</p>
<p>But where the change is most noticeable is on slow mobile devices on a 2G connection. Under these conditions the page loads approximately 10 seconds faster according to our simulated <a href="https://speedcurve.com/">SpeedCurve</a> tests. This large performance increase comes in part from the 16% reduction in overall page size, but also from the fact we have removed around 200 KB of CSS that no longer needs to be processed by the browser before the page is shown.</p>
<h3 id="lower-battery-power-usage">Lower battery power usage</h3>
<p>With over 44% of GOV.UK users accessing the website through a mobile device, pages need to render quickly and efficiently. Mobile devices are often CPU, memory and battery limited.</p>
<p>A page with a large amount of blocking data can have a huge impact on users with pre-2012 devices. A GOV.UK page will take longer to download due to the mobile connection speed. The CPU and RAM requirement will increase when processing the data and display content to the screen. These both have an impact on a users mobile battery life. The same principle applies to users on older desktop and laptop devices.</p>
<p>By optimising the GOV.UK frontend code, we are improving the experience for all users no matter what device they are using to access it.</p>
<h2 id="continuing-to-improve-govuk-frontend-performance">Continuing to improve GOV.UK frontend performance</h2>
<p>This is just the first stage in improving the GOV.UK frontend performance. The <a href="https://www.a2-type.co.uk/new-transport">font foundry</a> is working on a new font that promises to reduce the font weight by another 47% for WOFF2 (50% for WOFF).</p>
<p>We’ll also experiment with new browsers features like the <a href="https://www.zachleat.com/web/font-checklist/"><code class="highlighter-rouge">font-display</code> property and asset preloading</a>. By combining all these changes we’ll aim to reduce the load time and also the overall page weight for GOV.UK users. This will provide a better user experience for all users no matter how they access the site.</p>Matt HobbsOptimising the font loading method on GOV.UK to improve performance.Changing our testing requirements for Internet Explorer 8, 9 and 102018-06-26T12:09:00+00:002018-06-26T12:09:00+00:00https://nooshu.github.io/blog/2018/06/26/changing-our-testing-requirements-for-internet-explorer-8-9-and-10<hr />
<p><strong>Note</strong>: This post was originally posted on the <a href="https://gdstechnology.blog.gov.uk/">GDS Technology blog</a>. It can be <a href="https://gdstechnology.blog.gov.uk/2018/06/26/changing-our-testing-requirements-for-internet-explorer-8-9-and-10/">viewed here</a>.</p>
<hr />
<figure class="figure">
<img class="figure__image" src="/images/2018/06/ie-browser-upgrade.png" alt="Time to upgrade your old IE browser advert from Microsoft." />
</figure>
<p>Service team developers no longer need to test on Internet Explorer 8, 9 and 10 (old versions of IE) when following our ‘<a href="https://www.gov.uk/service-manual/technology/designing-for-different-browsers-and-devices">designing for different browsers and devices</a>’ guidance.</p>
<p>As users upgrade the devices they use to access the web, older browsers become less common. Your service must be universally accessible, but there are 3 big downsides to supporting older browsers once they have fallen out of use for most people. Below we explain why we’ve made this decision.</p>
<h2 id="why-we-no-longer-require-teams-to-test-older-versions-of-internet-explorer">Why we no longer require teams to test older versions of Internet Explorer</h2>
<h3 id="security-issues">Security issues</h3>
<p><a href="https://www.microsoft.com/en-gb/windowsforbusiness/end-of-ie-support">Microsoft stopped regularly supporting older versions of Internet Explorer on 12 January 2016</a>. Since then, old IE browsers have received a few critical security updates but they don’t support all the latest security standards. This means these browsers are not secure for users and departments who continue to use them.</p>
<p>The National Cyber Security Centre (NCSC) currently recommends using the <a href="https://www.ncsc.gov.uk/guidance/tls-external-facing-services#UsingTLStoprotectdata-Combination1oftheSuiteBprofileforTLS">Transport Layer Security 1.2 protocol</a> to provide privacy between communicating applications and their users, and also between services. However, TLS 1.2 is either not supported or not enabled by default in old IE browsers (depending on operating system in use). This means that certain services, such as <a href="https://www.payments.service.gov.uk/">GOV.UK Pay</a> which require a minimum of TLS 1.2 due to PCI compliance (PCI DSS), won’t work. Other services, such as <a href="https://www.notifications.service.gov.uk/">GOV.UK Notify</a> and <a href="https://www.cloud.service.gov.uk/">GOV.UK PaaS</a>, also plan to require TLS 1.2 in the future.</p>
<p>Older versions of Internet Explorer don’t support the latest HTML5 features either. This means they won’t benefit from security features like:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Punycode">Punycode identification</a></li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS">Cross-Origin Resource Sharing (CORS)</a></li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP">Content Security Policy (CSP)</a></li>
<li><a href="https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security">HTTP Strict Transport Security (HSTS)</a></li>
</ul>
<p>These features are all available in <a href="https://www.techopedia.com/definition/31094/evergreen-browser">‘evergreen’ browsers</a> like Microsoft Edge, Google Chrome and Mozilla Firefox. Browsers like these update automatically without user interaction.</p>
<h3 id="poor-user-experience">Poor user experience</h3>
<p>Browser technology has evolved and no matter how much user testing is done, people with old IE browsers will have a poor user experience when accessing government websites. Although the pages will work thanks to the <a href="https://www.gov.uk/service-manual/technology/using-progressive-enhancement">progressive enhancement techniques we use</a>, old browsers will not provide as good an experience as with modern browsers.</p>
<p>Webpages take longer to load and may even look broken in many cases, due to a lack of support for the latest browser standards. If the very small percentage of users (or the IT teams in their departments) relying on old browsers make an upgrade, their overall experience of government services will improve.</p>
<h3 id="code-complexity">Code complexity</h3>
<p>Supporting older browsers creates code complexity for developers. The codebase is harder to maintain which can lead to technical debt, making things harder for future developers who inherit the project. The additional testing required to enable a service to work in these older browsers also adds to development timelines. This extra development effort costs time and money which isn’t cost-effective when old IE has known security and performance issues.</p>
<h2 id="what-this-change-means-for-govuk-users">What this change means for GOV.UK users</h2>
<p>The number of users visiting GOV.UK using old IE browsers has fallen dramatically in the past 12 months, from 1.11% to 0.16% of total users in May 2018.</p>
<p>This means the number of users affected by the change to the guidance is small, compared to the overall traffic we receive. And, according to our <a href="https://www.gov.uk/service-manual/technology/designing-for-different-browsers-and-devices#understanding-the-table">Service Manual guidelines</a> and Google Analytics data, we should focus on supporting the browsers and devices that are in use.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/06/ie8-10-usage-govuk.png" alt="Graph showing the decline in old Internet Explorer usage on GOV.UK." />
</figure>
<p>The change to our guidance doesn’t mean that GOV.UK pages will stop working on older browsers. We still recommend that your service teams use the <a href="https://www.gov.uk/service-manual/technology/using-progressive-enhancement">progressive enhancement development</a> approach. By doing this we can help to make sure that the small number of users who use old IE browsers will still be able to access government content but some services may have limited functionality.</p>
<p>When users use old browsers to access GOV.UK they will also see a banner with a link to <a href="https://www.gov.uk/help/browsers">a page to help them upgrade</a>.</p>
<p>If you provide internal-facing services, you should look at your analytics data and check the percentage of users that use these older browsers. You can then make an informed decision about whether to support or drop old IE from your browser matrix.</p>
<p>Continuing to support these older browsers beyond their life cycle ultimately costs the taxpayer more time and money. This money would be better spent on creating and improving services for the other 99.8% of users who access GOV.UK and services that sit beneath it. It’s time to say goodbye to these legacy browsers and move towards a more standards-compliant future.</p>
<p>To make sure you stay up to date with all the latest developments, you can sign up to alerts from the <a href="https://gdstechnology.blog.gov.uk/subscribe/">GDS Technology blog</a>.</p>Matt HobbsOld versions of Internet Explorer are in decline, so let's reconsider the testing requirements.Cloudless Chrome - Broken CSS filters2018-05-18T12:09:00+00:002018-05-18T12:09:00+00:00https://nooshu.github.io/blog/2018/05/18/Cloudless-chrome<p>A month or so ago I happened to open this very website in the latest version of Chrome (65 at the time) and I noticed something was a little bit off. The animating clouds in the header of the website were very broken. Considering the code for this part of the site hasn’t changed since I first built it, this pointed to the fact that this change hadn’t come from me.</p>
<p>As you can see in the images below, there was quite a rendering difference between Firefox and Chrome.</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/05/clouds-in-firefox.png" alt="Example of the clouds in Firefox browser" />
<figcaption class="figure__caption">Animated SVG clouds with a slight blur in Firefox.</figcaption>
</figure>
<figure class="figure">
<img class="figure__image" src="/images/2018/05/clouds-in-chrome.png" alt="Example of the broken clouds in Chrome browser" />
<figcaption class="figure__caption">There are supposed to be some clouds here, but Chrome is having issues.</figcaption>
</figure>
<h2 id="what-happened">What happened?</h2>
<p>Well as you may (or may not) know all modern web browsers are what is known as “evergreen browsers”. This is defined as:</p>
<blockquote>
<p>…browsers that are automatically upgraded to future versions, rather than being updated by distribution of new versions from the manufacturer, as was the case with older browsers.</p>
</blockquote>
<p>So the Chrome browser had automatically updated and in doing so a regression had been introduced. Unfortunately this is one negative point around auto-updating browsers that require no user interaction. The update will happen, something will break and the user will have no idea what changed. Luckily this doesn’t happen very often as browser vendors have a very extensive testing process before a new version hits the stable channel. It takes approximately 10 weeks from a commit entering the Canary channel to land on the stable channel. You can see what the Chrome team have in the pipeline on the <a href="https://www.chromestatus.com/features">ChromeStatus website</a>.</p>
<h2 id="css-filter-broke-the-clouds">CSS filter broke the clouds</h2>
<p>So what actually broke? It turned out a single line of CSS was causing the issue:</p>
<div class="language-css highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">.cloud-class-example</span> <span class="p">{</span>
<span class="err">//...</span>
<span class="nl">filter</span><span class="p">:</span> <span class="n">blur</span><span class="p">(</span><span class="m">2.2px</span><span class="p">)</span>
<span class="p">//...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Applying the <code class="highlighter-rouge">blur()</code> CSS filter to the cloud SVG was breaking the rendering. After a little investigation it turns out that applying any CSS filter seemed to break the rendering, it wasn’t just tied to <code class="highlighter-rouge">blur()</code>.</p>
<p>Looking through the <a href="https://bugs.chromium.org/">Chromium bugs</a> for anything related to “CSS filter”, two bugs are listed that looked to be very similar to the issues I’ve been seeing:</p>
<ul>
<li><a href="https://bugs.chromium.org/p/chromium/issues/detail?id=830242">830242 - Chrome65 CSS filter rendering exceptions</a></li>
<li><a href="https://bugs.chromium.org/p/chromium/issues/detail?id=804164">804164 - Some combinations of CSS filters cause complete rendering failures</a></li>
</ul>
<p>Reading through these bug reports it turns out that this bug is very specific to OSX version 10.11 with Intel video cards. So only a certain number of users will be seeing this issue. Checking my computer specifications, sure enough 10.11 with an Intel video card!</p>
<h2 id="examples">Examples</h2>
<p>For those wondering if you’re affected by this bug I’ve created a very basic test page that can be viewed on <a href="https://www.sassmeister.com/gist/1a6ff0dfd6ec47bbceb07c5841060f29">Sassmeister</a> (<a href="https://gist.github.com/Nooshu/1a6ff0dfd6ec47bbceb07c5841060f29">Gist</a>).</p>
<p>Here’s a screenshot from Firefox of what the rendering should look like:</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/05/firefox-example.png" alt="Firefox rendering the example" />
<figcaption class="figure__caption">Firefox browser renders the example as expected.</figcaption>
</figure>
<p>Compare it to how it renders in Chrome 66. Very broken!</p>
<figure class="figure">
<img class="figure__image" src="/images/2018/05/chrome-example.png" alt="Chrome rendering the example" />
<figcaption class="figure__caption">Chrome 66 looks really broken, with some clouds not even being rendered.</figcaption>
</figure>
<p>In the example code you will see I tried a number of permutations to see if it was a combination of the animation and the filter, or if it was only a SVG rendering issue related to the filter. It turns out that it has nothing to do with the animation or SVG’s. The clouds are still broken on non-animated assets and non-SVG content (<code class="highlighter-rouge">&lt;div&gt;</code>).</p>
<h2 id="solution">Solution</h2>
<p>So what’s the solution for this issue you may ask? Well aside from upgrading your OS from OSX 10.11 or removing the CSS filtering from the element, there really isn’t a solution to this issue at the moment. Oh no! you may shout. But do not despair, the Google Chrome team have it covered. You will be pleased to hear that in the latest Chrome Canary (68 at the time of writing) all the examples render fine!</p>
<p>The list of browsers with this specific bug for OSX 10.11 with Intel GPU’s are:</p>
<ul>
<li>Chrome 64 - not broken</li>
<li>Chrome 65 - <strong>broken</strong></li>
<li>Chrome 66 - <strong>broken</strong> (current stable channel)</li>
<li>Chrome 67 - <strong>broken</strong> (current beta channel)</li>
<li>Chrome 68 - not broken (current Canary channel)</li>
</ul>
<p>Chrome 68 has a planned release date of July 2018. This is when it should hit the stable channel, at which point I expect this issue to rectify itself very quickly for the small number of users it affects.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Todays “evergreen browsers” will auto-update without the need for any user interaction and in doing so this can sometimes introduce regressions. The average user won’t understand the origin of this regression since it will look like nothing has changed.</p>
<p>Is this a bad? Sure it could be! But if you compare it to the web when we didn’t have “evergreen browsers” and we relied on users or admins to update the browser version (IE8 anyone?), we are in a much better place now. New features and bug fixes are rolled out quickly, meaning a safer and more exciting web for users and developers alike!</p>
<p>You will be pleased to hear that all major browser vendors now have auto-updating browsers (even Microsoft Edge!).</p>Matt HobbsRecently something has gone slightly awry with the clouds in the header on Chrome. I wonder what it could be?Workbox - Generate a service worker with ease2018-03-02T12:09:00+00:002018-03-02T12:09:00+00:00https://nooshu.github.io/blog/2018/03/02/workbox-generate-with-ease<p>Back in July 2016 I wrote a blog post about how to <a href="/blog/2016/07/22/implementing-a-service-worker/">implement a service worker</a>. At the time the technology was still fairly new, with only Firefox, Chrome, Opera and Samsung Internet having <a href="https://beta.caniuse.com/#search=service%20worker">enabled the API</a> in their browsers. It’s amazing what a difference 18 months can make in the Frontend world. WebKit <a href="https://webkit.org/blog/8084/release-notes-for-safari-technology-preview-48/">announced in January</a> that service workers would be supported in Safari Technology Preview 48. Closely followed by <a href="https://blogs.windows.com/msedgedev/2018/02/06/welcoming-progressive-web-apps-edge-windows-10/#bZSi1lwOHSWiSppM.97">Microsoft announcing Edge 17 will also support them</a>. The future looks bright for service workers so we’d better get used to creating them! So the question you may be asking is how do I create one?</p>
<h2 id="the-old-way-copy--paste--hope">The old way (copy / paste / hope)</h2>
<p>The old way seemed very much a case of read the documentation, copy a script, modify for your own needs then hope you didn’t break anything when you deploy it to your users. There’s nothing wrong with this method and it is a great way to learn a new technology (I do it all the time!). But this isn’t a good strategy to use when it comes to service workers.</p>
<p>With a service worker, if you aren’t careful you could inadvertently break your website completely for your users. If this were to happen they would have no idea why or even how to fix it. Imagine a service worker being installed on a users machine that is broken in some way. Say for example all requests are swallowed by the service worker once it is activated. When the user revisits your site they won’t see a homepage. In fact they would most likely only get a standard browser error page, but only for your website. This is bad! But what is worse is that since the user can no longer get to your website, they can also no longer receive service worker updates to fix the issue! Now you may be thinking ‘they can just clear the cache’. Afraid not, even if they clear the browser cache the service worker will still persist. You’ve inadvertently created a <a href="https://www.youtube.com/embed/CPP9ew4Co0M?rel=0">zombie service worker</a> on the users machine that they can only kill if they know how to use the DevTools or update the device.</p>
<p>Alexander Pope spoke about this topic in his 25 minute presentation at JSConf EU 2017:</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/CPP9ew4Co0M?rel=0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>Scary stuff huh! So what’s the solution? Well you could be very careful and test your code multiple times on multiple machines before releasing to production… but you are already doing that anyway aren’t you? So that may not help you in this instance. It simply isn’t possible to test all possible permutations across all devices. Another more viable solution to consider is simply abstract the problem away. This is where <a href="https://developers.google.com/web/tools/workbox/">Workbox</a> comes to the rescue.</p>
<h2 id="the-new-way-workbox">The new way (Workbox)</h2>
<p><a href="https://developers.google.com/web/tools/workbox/">Workbox</a> is a collection of libraries that you can use to generate your service workers in a safe and well tested manner. If you are looking to add offline functionality or improve loading performance for repeat users, there really isn’t a simpler way to do it. It will even make deployment much easier because you can even integrate Workbox into your development workflow using npm (workbox-cli), Gulp or Webpack.</p>
<p>I recently removed my own implementation of a service worker for this blog and replaced it with one generated by Workbox. Now every time I run gulp to generate the static website (I use <a href="https://jekyllrb.com/">Jekyll</a>), Workbox examines the resulting HTML and assets looking for byte-level differences from the previous versions. If differences are found it will generate a new service worker file (sw.js) with the relevant files revision hash updated. The update cycle will then be triggered on the clients machine and the service worker will cache the modified assets. You are now no longer required to manually manage your cached assets and service worker script!</p>
<h2 id="example-setup">Example setup</h2>
<p>To setup Workbox with Gulp it is fairly straightforward. The examples below are what I have used for this site.</p>
<h3 id="config">Config</h3>
<p>This is the config used by Workbox to define what is cached and where the final service worker file (sw.js) is generated. Notice how there is a Stale While Revalidate request strategy used with the Google Font. This means the resource is requested from both the cache and the network in parallel, then the service worker immediately responds with the cached version. If there is an updated version of the resource it is swapped out in the background, with no impact on a users browsing experience. This strategy is great for non-critical page assets like favicons and avatars. It also displays another powerful feature of Workbox. You can easily define different request strategies for different assets in the config and it will handle the implementation for you.</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="code"><pre><span class="c1">// workbox-cli-config.js</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">globDirectory</span><span class="p">:</span> <span class="s1">'_site'</span><span class="p">,</span>
<span class="na">globPatterns</span><span class="p">:</span> <span class="p">[</span>
<span class="s1">'images/app-shell/*'</span>
<span class="p">],</span>
<span class="na">swDest</span><span class="p">:</span> <span class="s1">'_site/sw.js'</span><span class="p">,</span>
<span class="na">clientsClaim</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">skipWaiting</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">runtimeCaching</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="na">urlPattern</span><span class="p">:</span> <span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="s1">'https://fonts.googleapis.com/css?family=Dosis|Open+Sans'</span><span class="p">),</span>
<span class="na">handler</span><span class="p">:</span> <span class="s1">'staleWhileRevalidate'</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">};</span></pre></td></tr></tbody></table></code></figure>
<h3 id="gulp">Gulp</h3>
<p>For reference I have my gulp tasks split into separate files for easy maintenance. There really isn’t much going on in this file. We require Workbox, point it to the config defined above and watch for success / errors.</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="code"><pre><span class="c1">// generate-service-worker.js</span>
<span class="kd">const</span> <span class="nx">gulp</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'gulp'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">workbox</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'workbox-build'</span><span class="p">);</span>
<span class="kd">function</span> <span class="nx">generateServiceWorker</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">workbox</span><span class="p">.</span><span class="nx">generateSW</span><span class="p">(</span>
<span class="nx">require</span><span class="p">(</span><span class="s1">'../workbox-cli-config'</span><span class="p">)</span>
<span class="p">).</span><span class="nx">then</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">info</span><span class="p">(</span><span class="s1">'Service worker generation completed.'</span><span class="p">);</span>
<span class="p">}).</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">warn</span><span class="p">(</span><span class="s1">'Service worker generation failed: '</span> <span class="o">+</span> <span class="nx">error</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="nx">gulp</span><span class="p">.</span><span class="nx">task</span><span class="p">(</span><span class="s1">'generate-service-worker'</span><span class="p">,</span> <span class="nx">generateServiceWorker</span><span class="p">);</span></pre></td></tr></tbody></table></code></figure>
<p>I’ve had to create an additional gulp task to copy the service worker files from the <code class="highlighter-rouge">_site</code> directory (where the static site is generated by gulp), back into the project root. This task is required because GitHub pages doesn’t generate a <code class="highlighter-rouge">_site</code> directory when deploying the site. Once the file is moved and committed into the project root it will then be deployed by GitHub pages like any other file.</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="code"><pre><span class="c1">// copy-sw-code-to-root.js</span>
<span class="kd">const</span> <span class="nx">gulp</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'gulp'</span><span class="p">);</span>
<span class="kd">function</span> <span class="nx">copySwCodeToRoot</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">gulp</span><span class="p">.</span><span class="nx">src</span><span class="p">([</span><span class="s1">'_site/sw.js'</span><span class="p">,</span> <span class="s1">'_site/workbox*'</span><span class="p">])</span>
<span class="p">.</span><span class="nx">pipe</span><span class="p">(</span><span class="nx">gulp</span><span class="p">.</span><span class="nx">dest</span><span class="p">(</span><span class="s1">'./'</span><span class="p">));</span>
<span class="p">}</span>
<span class="nx">gulp</span><span class="p">.</span><span class="nx">task</span><span class="p">(</span><span class="s1">'copy-sw-code-to-root'</span><span class="p">,</span> <span class="nx">copySwCodeToRoot</span><span class="p">);</span></pre></td></tr></tbody></table></code></figure>
<h3 id="output">Output</h3>
<p>What comes out the end is clean and simple auto-generated sw.js file that you can <a href="https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers#Enter_service_workers">register in the usual manner</a>. Note the importScripts() at the top of the generated file. This is loading a production version of Workbox.js. This is the file that does all the heavy lifting for us so we don’t need to get involved in all the details of the service worker setup. These implementation details have been abstracted away into well tested external script that we don’t need to worry about. Very useful!</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
</pre></td><td class="code"><pre><span class="c1">// sw.js</span>
<span class="nx">importScripts</span><span class="p">(</span><span class="s1">'workbox-sw.prod.v2.1.2.js'</span><span class="p">);</span>
<span class="cm">/**
* DO NOT EDIT THE FILE MANIFEST ENTRY
*
* The method precache() does the following:
* 1. Cache URLs in the manifest to a local cache.
* 2. When a network request is made for any of these URLs the response
* will ALWAYS comes from the cache, NEVER the network.
* 3. When the service worker changes ONLY assets with a revision change are
* updated, old cache entries are left as is.
*
* By changing the file manifest manually, your users may end up not receiving
* new versions of files because the revision hasn't changed.
*
* Please use workbox-build or some other tool / approach to generate the file
* manifest which accounts for changes to local files and update the revision
* accordingly.
*/</span>
<span class="kd">const</span> <span class="nx">fileManifest</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s2">"url"</span><span class="p">:</span> <span class="s2">"images/app-shell/bubble.svg"</span><span class="p">,</span>
<span class="s2">"revision"</span><span class="p">:</span> <span class="s2">"cd8f3e69287067840832444472f3aece"</span>
<span class="p">},</span>
<span class="c1">//...</span>
<span class="p">];</span>
<span class="kd">const</span> <span class="nx">workboxSW</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">self</span><span class="p">.</span><span class="nx">WorkboxSW</span><span class="p">({</span>
<span class="s2">"skipWaiting"</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="s2">"clientsClaim"</span><span class="p">:</span> <span class="kc">true</span>
<span class="p">});</span>
<span class="nx">workboxSW</span><span class="p">.</span><span class="nx">precache</span><span class="p">(</span><span class="nx">fileManifest</span><span class="p">);</span>
<span class="nx">workboxSW</span><span class="p">.</span><span class="nx">router</span><span class="p">.</span><span class="nx">registerRoute</span><span class="p">(</span><span class="sr">/https:</span><span class="se">\/\/</span><span class="sr">fonts.googleapis.com</span><span class="se">\/</span><span class="sr">css</span><span class="se">?</span><span class="sr">family=Dosis|Open+Sans/</span><span class="p">,</span> <span class="nx">workboxSW</span><span class="p">.</span><span class="nx">strategies</span><span class="p">.</span><span class="nx">staleWhileRevalidate</span><span class="p">({}),</span> <span class="s1">'GET'</span><span class="p">);</span></pre></td></tr></tbody></table></code></figure>
<h2 id="conclusion">Conclusion</h2>
<p>I’ve only scratched the surface of what Workbox can do. Take a look at the <a href="https://developers.google.com/web/tools/workbox/reference-docs/latest/">documentation</a> and <a href="https://codelabs.developers.google.com/codelabs/workbox-lab/index.html?index=..%2F..%2Findex#0">tutorials</a> available for more information about the full feature set of Workbox. Version 3.0.0 beta 0 was <a href="https://github.com/GoogleChrome/workbox/releases/tag/v3.0.0-beta.0">released very recently</a>, with a whole new set of features. It is certainly a project to keep watch or <a href="https://github.com/GoogleChrome/workbox">star on GitHub</a>.</p>
<p>Progressive Web Apps and service workers are here to stay. You only need to take a look in your Chrome DevTools under “Applications &gt; Service Workers” to see how many websites you browse have already installed a service worker on your machine. If treated with care and implemented correctly they can significantly boost your site performance and offer your users some really cool features. But if the implementation goes wrong… well you may never see those users again! Using Workbox to generate the service worker for your Progressive Web App should be high on your priority list.</p>Matt HobbsService workers are all the rage at the moment, but how do we simplify their creation?We’ve added Samsung Internet to our browser testing recommendations2018-02-26T12:09:00+00:002018-02-26T12:09:00+00:00https://nooshu.github.io/blog/2018/02/26/weve-added-samsung-internet-to-browser-testing-recommendations<hr />
<p><strong>Note</strong>: This post was originally posted on the <a href="https://gdstechnology.blog.gov.uk/">GDS Technology blog</a>. It can be <a href="https://gdstechnology.blog.gov.uk/2018/02/26/weve-added-samsung-internet-to-our-browser-testing-recommendations/">viewed here</a>.</p>
<hr />
<figure class="figure">
<img class="figure__image" src="/images/2018/02/samsung-internet-sm.jpg" alt="GOV.UK displayed on Samsung Internet." />
</figure>
<p>We update our browser <a href="https://www.gov.uk/service-manual/technology/designing-for-different-browsers-and-devices#browsers-to-test-in">testing recommendations in the Service Manual</a> every 6 months. Samsung Internet is the latest addition to our list to test your service in.</p>
<h2 id="why-we-added-samsung-internet">Why we added Samsung Internet</h2>
<p>Samsung Internet is the default browser on Samsung devices and has <a href="http://gs.statcounter.com/browser-market-share/all/united-kingdom">2.71% of UK browser market share</a> and over 400 million active monthly users worldwide.</p>
<p>This browser now accounts for a significant share of traffic to GOV.UK. From December to January, 466,012 users visited GOV.UK using Samsung Internet, accounting for 3.2% of traffic. On GOV.UK’s 10 most-used service start pages, Samsung Internet accounts for between 1% and 5% of traffic.</p>
<h2 id="samsung-internet-and-chrome-for-android-are-different-and-more-changes-are-expected">Samsung Internet and Chrome for Android are different and more changes are expected</h2>
<p>Samsung Internet is one of several browsers for the Android operating system built on Chromium, <a href="https://www.chromium.org/">the open source code project</a> that’s also the basis for the Google Chrome browser.</p>
<p>Webpages and features on Samsung Internet and Google Chrome for Android function in a similar way but this might change as the browsers diverge. Google Analytics started to distinguish between Samsung Internet and Chrome for Android in August 2017 - previously, it had counted them as the same browser.</p>
<p>Some <a href="https://caniuse.com/#compare=and_chr+64,samsung+6.2">functional differences between the 2 browsers</a> are already apparent and web developers will need to consider slight differences in CSS, JavaScript and interface features when building for this browser mix.</p>
<p>In the future, there may be a fundamental difference in how pages render or function on Samsung Internet and Chrome for Android. That’s why we think testing this new, popular browser is important.</p>
<h2 id="having-more-browsers-around-is-better-for-everyone">Having more browsers around is better for everyone</h2>
<p>Diversity in the browser market is great for users. Competition between browsers helps to stimulate development of the web, as browsers are iterated to become faster, more secure and easier to use. We’ll keep monitoring use data to make sure GOV.UK keeps pace with the evolving browser space.</p>Matt HobbsThe Service Manual gets an update to add Samsung Internet to the testing matrix.Removing Accelerated Mobile Pages (AMP)2018-02-06T12:09:00+00:002018-02-06T12:09:00+00:00https://nooshu.github.io/blog/2018/02/06/removing-amp<p>Around 18 months ago I wrote a blog post called <a href="/blog/2016/08/07/accelerated-mobile-pages/">Accelerated Mobile Pages (AMP)</a> (imaginative I know!) about implementing AMP into this very website. At the time it was purely out of curiosity. I happened to see a new navigation item pop up in the left hand panel of Webmaster Tools and thought I would give it a try. So why am I now removing it? I’ve listed a few of my reasons below.</p>
<h2 id="original-purpose">Original Purpose</h2>
<p>The purpose of Google AMP is to improve the user experience of the web. It does this by enforcing a strict standard on a set of custom elements. These custom elements can then be highly optimised for speed. Speed is the key selling point when adopting AMP. Pages across the web are becoming bloated and slow to load and this is becoming a real problem on mobile devices which have a limited resources and data available to them. Web sites in 2018 have more code, larger images, more ads, more of everything! I’m just glad I’m no longer on a 56K modem, can you imagine the frustration?!</p>
<p>Now I must admit AMP pages load really quickly and one reason for this is a browser will <a href="https://medium.com/@cramforce/why-amp-html-does-not-take-full-advantage-of-the-preload-scanner-7e7f788aa94e">prerender the page</a> before a user even clicks the link. This is really handy if you are on an intermittent connection (like on the tube for example). Need a page to read before you head into the next tunnel? Look for the AMP logo in the search results and it will already be there for you to read, no more data required (for the “above-the-fold” visible content). I hear you say “But that is awesome! What’s not to like?”. Well for me it all comes down the finer details.</p>
<h2 id="amp-html-implementation">AMP HTML implementation</h2>
<p>The implementation wasn’t difficult, it was just a real pain to make sure everything was valid. I use a static site generator called <a href="https://jekyllrb.com/">Jekyll</a> which allows me to easily host this blog on GitHub pages. Unfortunately, because you are rendering from Markdown and not handcrafting HTML, you don’t have 100% control over the HTML that comes out the other end. So if some HTML that is generated isn’t considered valid AMP HTML, you either have to find a hack or you are out of luck. No Google AMP Cache for you! I mention one of these issues in the <a href="/blog/2016/08/07/accelerated-mobile-pages/#fixing-jekyll-rouge">original post</a> around the use of Jekyll Rouge.</p>
<h2 id="amp-js">AMP JS</h2>
<p>AMP is very (very) strict when it comes to JavaScript. No JavaScript to be served other than AMP JS. So no <a href="https://jquery.com/">jQuery</a>, no <a href="https://getbootstrap.com/docs/4.0/getting-started/javascript/">Bootstrap</a> or <a href="https://foundation.zurb.com/sites/docs/javascript.html">Foundation</a>, no nothing. The only JavaScript that is allowed to be served is AMP JS which is the library that powers AMP. Again note that it can only be served from Google, a self hosted version isn’t allowed.</p>
<p>So what is AMP JS? Well it is used to to transform the custom elements that Google has defined into elements that the browser understands. So why no other JavaScript allowed? Well the purpose stated is that to maximise page load speed, the browsers critical rendering path needs to be minimised. Adding other JavaScript will break this requirement if implemented incorrectly, so it is safer to remove all other JavaScript completely. Seems a little extreme! Is it true that AMP JS is all about speed? Not according to Maciej Cegłowski who rewrote the <a href="http://idlewords.com/amp_static.html">original AMP demo page</a> without using AMP JS and it turned out to be much smaller and much quicker.</p>
<p>So it isn’t really about speed. I fear it is more to do with tracking and locking you into the ecosystem.</p>
<h2 id="amp-cache-and-validity">AMP Cache and validity</h2>
<p>Once all your pages are valid you are good to go though… not quite. There have been a couple of occasions where I would see that a number of pages had suddenly become invalid due to the AMP specification changing. Now this isn’t a big deal in itself as the changes were fairly minor, but it does highlight the fact that you are essentially using a third-party product that is out of your control to display a large percentage of your website to your users, and what they say goes. In this case said third-party is Google, so what they really say goes!</p>
<p>For your content to be added to the AMP Cache it must validate. No exceptions. For a small site with a low number of pages like mine this isn’t a big deal. For websites with tens of thousands of pages, that’s potentially a lot of work! So if the specification changes and a 90% of your pages are suddenly invalid, best case scenario is it’s a quick 2 minute change to a template, or you could be trawling through hundreds of pages to make them revalidate.</p>
<h2 id="ecosystem">Ecosystem</h2>
<p>Assuming you have taken the plunge and decided to adopt AMP, you are now essentially locked into the Google ecosystem. AMP is open source, but it is primarily driven by Google.</p>
<p>So you’ve decided to take the plunge and use AMP. Here’s what to expect:</p>
<ul>
<li>loading AMP JS: the JavaScript that powers AMP. This can only be served from the Google servers, no self-hosting allowed.</li>
<li>to use AMP Cache: a lot of the speed gained from AMP is the fact that the pages will be served from a CDN; these CDN’s are run by Google.</li>
<li>a Google CDN URL when sharing content: Want to share a page you found that is using AMP with a friend? You will be most likely be sharing the Google CDN URL rather than content from the original author (NOTE: It looks like they <a href="https://amphtml.wordpress.com/2018/01/09/improving-urls-for-amp-pages/">fixed this very recently!</a>).</li>
</ul>
<p>So what do you get when you use all these Google technical toys? Well a big plus is you get preferential treatment from Google in terms of mobile search. Your content will be boosted up the search rankings above other content (on Google at least). Say you give it a try and realise it isn’t for you. How easy is it to remove? Well it might not be as <a href="https://shkspr.mobi/blog/2016/11/removing-your-site-from-amp/">simple as you think</a>.</p>
<p>I think you get the idea. Google is fully in control of the bus, you’re just along for the ride.</p>
<h2 id="creativity">Creativity</h2>
<p>I think the main thing I don’t like about AMP is the restrictions on <a href="https://www.ampproject.org/learn/about-how/">JavaScript</a> and <a href="https://www.ampproject.org/docs/guides/responsive/style_pages">CSS</a> usage. Now I understand why those restrictions and specifications are so strictly enforced as they are performance best practices, but it takes some of the adventure out of being a Web Developer. I like being able to add random JavaScript functionality that doesn’t come pre-packaged in a toolkit. It is part of the fun. But then again maybe I’m just the wrong audience for AMP.</p>
<p>I fully admit I magnified my creativity restrictions problem because I went all out in my AMP usage. You have two options in AMP usage:</p>
<p>1) Lay your AMP website next to your standard website HTML so they both exist in tandem.
2) Go all out and use AMP HTML for all breakpoints and only have a single version.</p>
<p>I went for option 2. Had I decided to create two separate versions of the website this would have given me the creative freedom to add little bits of extra functionality here and there where needed. Which leads me onto the next oddity that I find unsettling…</p>
<h2 id="duplicate-content">Duplicate content</h2>
<p>For years one of the key rules when it came to SEO was never duplicate content. It was SEO 101, if you duplicate content Google will penalise you. Google will assume the content has been copied from another website and rank you down into oblivion. Now suddenly we find ourselves duplicated all of our websites content and essentially maintaining two separate websites. To me that isn’t a good way to go, but maybe that’s just me and I’m starting to turn into an old man?</p>
<h2 id="conclusion">Conclusion</h2>
<p>In conclusion I see why Google AMP exists and I understand what Google is trying to achieve (I think). I just don’t like the fact that it is locking you into the Google ecosystem and creating another “walled garden” on the web. Google is already the most powerful company on the internet and Google AMP is only going to increase their dominance.</p>
<p>What would be nice is if all Web Developers take the best practices that AMP promotes in minimising the browsers critical rendering path, and apply them to their current sites. That way we get a fast, decentralised open web that isn’t dominated by a single company. I wouldn’t hold your breath waiting for that to happen though!</p>Matt HobbsAfter over a year of using AMP, it's time to back out and remove it.Opening Atlassian JIRA tickets with iTerm22017-01-10T12:09:00+00:002017-01-10T12:09:00+00:00https://nooshu.github.io/blog/2017/01/10/opening-atlassian-jira-tickets-with-iterm2<p>Happy New Year everyone! My first post of 2017 is going to be a very short, but it may just help you speed up your terminal workflow.</p>
<p>Two tools I use on a day to day basis are <a href="https://www.iterm2.com/">iTerm2</a> and <a href="https://www.atlassian.com/software/jira">JIRA from Atlassian</a>. iTerm2 is a terminal replacement app for OSX, and if you haven’t used it before you really should give it a try (I’ve mentioned it before in a previous <a href="/blog/2016/04/13/reasons-to-be-bashful/">blog post</a>). JIRA is the go to web application for bug tracking; it also works well with the rest of the <a href="https://www.atlassian.com/software">Atlassian suite</a>.</p>
<p>For a while I’ve wanted to be able to open tickets right from the terminal window. This would come in extremely helpful when looking at a <code class="highlighter-rouge">git log</code> output like the one below for example:</p>
<figure class="figure">
<img class="figure__image" src="/images/2017/01/simple-git-log-example.png" alt="Sample git log including Atlassian ticket references." />
</figure>
<p>Luckily iTerm2 comes with a whole host of features, one of which is called ‘Smart Selection’. You may not have noticed this before, but if you click multiple times on certain types strings they will be selected as a whole (email addresses, filesystem paths, URI’s). This is Smart Selection in action. iTerm2 looks at the string directly under the cursor, runs a set of matching Regex’s against it, and then allows you to run the relevant action against it.</p>
<p>We can use this feature to open the JIRA ticket page relating to the issue listed in our git logs like so:.</p>
<figure class="figure">
<img class="figure__image" src="/images/2017/01/iterm2-jira-demo.gif" alt="Animated screencast of Smart Selection in action." />
</figure>
<p>Adding this functionality is very simple. Under iTerm2 preferences go to ‘Profiles’, then the ‘Advanced’ tab. Here you will see the ‘Smart Selection’ edit button. See the images below for the sample settings I’ve used:</p>
<figure class="figure">
<img class="figure__image" src="/images/2017/01/adding-iterm2-smart-selection-rule.png" alt="Example of the smart selection rule in iTerm2." />
</figure>
<p>Add <code class="highlighter-rouge">^[project-ticket-prefix]-(\d+)</code> to the Regex rules, then edit the corresponding action and add Open URL.. with your JIRA URL. <em>Note</em>: This assumes the string starts with the ticket prefix, which is the case in my git logs. You may need to adjust this for your own setup.</p>
<figure class="figure">
<img class="figure__image" src="/images/2017/01/adding-iterm2-smart-selection-action.png" alt="Assigning a smart selection action in iTerm2." />
</figure>
<p>After that you should be able to right click on one of your tickets whenever they appear in the terminal and see the ‘Open JIRA Ticket’ option. Simple!</p>
<p>If you’re a ZSH user, you may also be interested in the <a href="https://github.com/robbyrussell/oh-my-zsh/tree/master/plugins/jira">JIRA plugin</a> that allows you to interact with your JIRA instance via the CLI. I’ve yet to use it myself, but it does look like it could be quite useful.</p>Matt HobbsIf you have a dig around inside the iTerm2 settings you will find lots to play with like 'Smart Selection'. We can use it to our advantage.A boilerplate module setup using Browserify, Babel, Mocha, and Chai2016-12-16T12:09:00+00:002016-12-16T12:09:00+00:00https://nooshu.github.io/blog/2016/12/16/browserify-boilerplate-module<p>Over the past few months I’ve been taking a look at a few of the ES2015 features that have landed in JavaScript language. There some excellent additions that will help JS developers write more succinct code. My personal favorites at the moment are <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Template_literals">Template literals</a> and <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Functions/Default_parameters">Default parameters</a>. Being able to specify a default parameter without a load of boilerplate code at the top of a function is a real win, and no more concatenating strings when you need to include a variable.</p>
<p>So while experimenting with the new features, I figured I’d also bring a few other technologies together to build a very simple module boilerplate, that I could then use on future projects.</p>
<h2 id="purpose-of-the-boilerplate">Purpose of the boilerplate</h2>
<p>The initial purpose was to experiment with the ES2015 <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/import">import statement</a>, but this then branched out to include Babel, and Browserify. Once those were implemented, I figured I would also roll in Mocha, Chai, and jQuery.</p>
<p>There was a big question in my head on how these could all work together to create a simple workflow, which could then be built upon in the future. My checklist of what I wanted to achieve with the experiment was:</p>
<ul>
<li>Ability to use ES2015+ features</li>
<li>Split the code into separate modules, for easier maintenance and testability</li>
<li>Be able to bundle the JavaScript in an efficient manor</li>
<li>Work directly with the DOM using the querySelector methods, or even jQuery since I sometimes require old IE support</li>
<li>Be testable using a popular testing framework</li>
</ul>
<h2 id="technologies">Technologies</h2>
<p>A list of the technologies used and their purpose is listed below:</p>
<h3 id="es2015-the-language-formally-known-as-es6-formally-known-as-harmony">ES2015 (the language formally known as ES6, formally known as Harmony)</h3>
<p>ES2015 is the latest iteration of the JavaScript language. The year is now being used in the name, so it is easy to distinguish when the changes in the language were added (in theory!). So as you would expect there’s also an ES2016, and ES2017 in the pipeline. These latest iterations add a whole host of useful language features and improvements to make it easier to write JavaScript. The full ES2015 language specification can be found <a href="http://www.ecma-international.org/ecma-262/6.0/">here</a></p>
<h3 id="babel">Babel</h3>
<p>The problem with introducing new language features on the web platform is you still need to support older browsers. As the saying goes, you can’t break the web. Because of this, nobody wants to write some lovely ES2015 code, and then have to write another version just for older browsers. Older browsers won’t know how to interpret these new features, and the page will break.</p>
<p>This is where <a href="https://babeljs.io/">Babel</a> comes to the rescue. Babel is a ES2015 compiler, that will transform the latest features back into ES5 compatible code so older browsers will still work. It even has the ability to polyfill methods that older browsers don’t support; so there’s no reason not to start learning and using ES2015 today.</p>
<h3 id="mocha--chai">Mocha &amp; Chai</h3>
<p><a href="https://mochajs.org/">Mocha</a> is a very popular JavaScript testing framework that runs on Node.js. It allows a developer to write and run a set of tests across their JavaScript code, and accurately report the results. Any tests that fail can then be debugged and fixed.</p>
<p><a href="http://chaijs.com/">Chai</a> is a BDD / TDD assertion library that can be used with any testing framework, but is often used in conjunction with Mocha. It comes with a whole set of API’s that allow you to write your tests in whatever way you prefer (Should, Expect, Assert).</p>
<p>Using both libraries together can be a real life-saver when it comes to refactoring code. It is very quick and easy to see when new code changes have broken functionality (assuming you have written your tests correctly!).</p>
<h3 id="jquery">jQuery</h3>
<p>I don’t think I really need to explain this one do I? Everyone knows jQuery, I’m sure it has saved most of us from cross-browser hell in the past! I have included it here because I often have projects that still require IE8 support. In my opinion it is much easier to include jQuery, than pick out a host of micro-libs to do the same job, with all their custom API’s and documentation.</p>
<h3 id="browserify">Browserify</h3>
<p><a href="http://browserify.org/">Browserify</a> is the “glue” that brings all of these technologies together. Browserify allows you to write code like you would in Node.js (using the require() method). This then gets bundled up into a single JavaScript file that the browser can execute. Browserify deals with all the code dependencies for you, and in the case of this demo also compiles ES2015 back to ES5 using Babel mentioned above.</p>
<h2 id="simple-module-example">Simple module example</h2>
<p>The code below shows a very simple example on how modules are imported into the main index.js file, then Browserify does the rest for you. The init() method kicks off the DOM interaction in module 1, and a sample unique array is being imported in from module 2.</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="code"><pre><span class="c1">// pull in everything from the sample module using a path</span>
<span class="k">import</span> <span class="o">*</span> <span class="k">as</span> <span class="nx">moduleName</span> <span class="k">from</span> <span class="s1">'./modules/sample-module'</span><span class="p">;</span>
<span class="c1">// pull in single variable from a module using browserify alias (browserify.js)</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">uniqueArray</span> <span class="p">}</span> <span class="k">from</span> <span class="s1">'sample-module-2'</span><span class="p">;</span>
<span class="c1">// start the module, pass any options we need</span>
<span class="nx">moduleName</span><span class="p">.</span><span class="nx">init</span><span class="p">({</span><span class="na">exampleSetting</span><span class="p">:</span> <span class="s1">'String Value'</span><span class="p">});</span>
<span class="c1">// use a specific method in the module</span>
<span class="kd">const</span> <span class="nx">addResult</span> <span class="o">=</span> <span class="nx">moduleName</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span> <span class="p">);</span>
<span class="kd">const</span> <span class="nx">minusResult</span> <span class="o">=</span> <span class="nx">moduleName</span><span class="p">.</span><span class="nx">minus</span><span class="p">(</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">10</span> <span class="p">);</span>
<span class="c1">// results from module 1</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span> <span class="nx">addResult</span><span class="p">,</span> <span class="nx">minusResult</span> <span class="p">);</span>
<span class="c1">// results from module 2</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span> <span class="nx">uniqueArray</span> <span class="p">);</span></pre></td></tr></tbody></table></code></figure>
<p>The functionality of each module is hidden away in separate files, allowing you to easily modularise you code. Using this setup, it is easy to create private variables and methods inside the modules. A developer then only need expose what is required to the wider application using the <code class="highlighter-rouge">export</code> keyword.</p>
<p>I’ve created a very simple sample page you can view using the <code class="highlighter-rouge">grunt</code> command. Grunt compiles everything and then launches a small BrowserSync http server that can be used to view the page. For more information on how to run the code see the <a href="https://github.com/Nooshu/browserify-module-boilerplate">Github readme</a>. <strong>Note</strong>: If you prefer to use Gulp instead of Grunt, this is fully possible. I simply used Grunt in the demo as I already had most of the code written from previous projects. This allowed me to focus on the JavaScript, rather than workflow tooling.</p>
<h2 id="issues">Issues</h2>
<p>There was a fair bit of trial and error with the initial setup to get it working as I wanted. My main issue occurred when it came to implementing Mocha, Chai, and jQuery. Testing jQuery in a Node environment became a real pain because of two errors I ran into: <code class="highlighter-rouge">ReferenceError: $ is not defined</code> and <code class="highlighter-rouge">Error: jQuery requires a window with a document</code>.</p>
<p>The first, <code class="highlighter-rouge">$ no defined</code>, was easily solved. In the browser environment I had jQuery loaded as a global object via its own script. You then tell Browserify this is the case using <code class="highlighter-rouge">"jquery": "global:jQuery"</code> in the package.json file. This of course didn’t work in Node since it isn’t a browser environment, so the script wasn’t loaded. Adding <code class="highlighter-rouge">import $ from 'jquery';</code> inside the module fixed that.</p>
<p>The second, <code class="highlighter-rouge">jQuery requires a window with a document</code>, took a good few hours to fix. I tried everything I could find on Stack Overflow including <a href="https://github.com/tmpvar/jsdom">jsdom</a> and <a href="https://github.com/rstacruz/mocha-jsdom">mocha-jsdom</a>; But no matter what setup I tried, the error still occurred. In the end it was installing <a href="https://github.com/rstacruz/jsdom-global">jsdom-global</a> that fixed it. Install it, create yourself a <code class="highlighter-rouge">mocha.opts</code> file for your the custom settings, and you are ready to go.</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="code"><pre><span class="nx">describe</span><span class="p">(</span> <span class="s1">'Example test description'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
<span class="kd">let</span> <span class="nx">$</span><span class="p">;</span>
<span class="nx">before</span><span class="p">(</span> <span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
<span class="nx">$</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span> <span class="s1">'jquery'</span> <span class="p">);</span>
<span class="p">}</span> <span class="p">);</span>
<span class="c1">// your tests go here</span>
<span class="p">}</span> <span class="p">);</span></pre></td></tr></tbody></table></code></figure>
<p>Then you can either run Mocha from the command line using <code class="highlighter-rouge">npm run test</code> with the following settings in your package.json:</p>
<figure class="highlight"><code class="language-javascript" data-lang="javascript"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="code"><pre><span class="s2">"scripts"</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"test"</span><span class="p">:</span> <span class="s2">"./node_modules/mocha/bin/mocha ./src/js/modules/test --reporter list --opts ./src/js/modules/test/mocha.opts"</span>
<span class="p">},</span></pre></td></tr></tbody></table></code></figure>
<p>Or what I prefer to do is run the tests directly in my IDE. <a href="https://www.jetbrains.com/webstorm/">JetBrains WebStorm</a> is my editor of choice, just remember you may need to point your IDE towards the <code class="highlighter-rouge">mocha.opts</code> file in the test settings.</p>
<figure class="figure">
<img class="figure__image" src="/images/2016/12/webstorm-javascript-testing.png" alt="Using Mocha for JavaScript testing inside JetBrains WebStorm." />
</figure>
<h2 id="conclusion">Conclusion</h2>
<p>This is my first iteration of a module system I plan to use on new projects going forwards, and it will continue to evolve to fit particular project requirements as they are discovered. You can find all the code, along with the very simple example on <a href="https://github.com/Nooshu/browserify-module-boilerplate">GitHub</a>. Feel free to use it if you like, or even suggest improvements. I’m always open to constructive criticism so raise an issue or get in contact with me via the contact page.</p>Matt HobbsTime to have a little play around with the future of JavaScript, ES2015 (and beyond!)Accelerated Mobile Pages (AMP)2016-08-07T12:09:00+00:002016-08-07T12:09:00+00:00https://nooshu.github.io/blog/2016/08/07/accelerated-mobile-pages<p>One of the advantages of having a blog, especially one built using a static site generator like <a href="https://jekyllrb.com/">Jekyll</a>, is it’s extremely easy to craft the HTML and have it compile as you’d expect it too. No CMS rendering out a load of HTML / CSS / JS that most likely isn’t required and just bloats the page. Having such a site also allows me to experiment with technology that would require quite a major refactor on a larger client build. Recently I have been looking at <a href="https://www.ampproject.org/">Accelerated Mobile Pages</a> (AMP). I have implemented the changes required to support it across the blog… did you notice?</p>
<h2 id="what-is-amp">What is AMP?</h2>
<p>Accelerated Mobile Pages is an open source project that aims to improve a user experience while browsing web content on a mobile device. It does this by taking standard HTML, and limiting its usage to a specific set of functionality that is clearly defined in the AMP spec documentation. The reason for this limitation is to improve performance on mobile devices, by only allowing current best practices across a wide range of component patterns. So for example, there are strict rules around the use of script and style tags. Too many of each could potentially slow down a page on a mobile device by blocking page rendering. AMP comes with its own validator that will report these errors so they can be corrected.</p>
<p>AMP comes in three parts:</p>
<ul>
<li>AMP HTML</li>
<li>AMP JS</li>
<li>Google AMP Cache</li>
</ul>
<p>AMP HTML is much like regular HTML, only extended slightly with a set of custom AMP properties. Some regular HTML tags have been replaced with AMP specific tags, allowing for common development patterns to be easily implemented in a performant manor. See the <a href="https://github.com/ampproject/amphtml/blob/master/spec/amp-html-format.md">AMP HTML Spec</a> for more details.</p>
<p>AMP JS is the glue that binds everything together. It is a library that adds all the custom elements mentioned above, and implements all of the AMP best practices. A key feature of the library includes allowing only asynchronous scripts, so nothing can block the DOM. Another large change (for me anyway) was all resources must be sized statically, meaning all images must have a width and height attribute. Including this width and height attribute allows the browser to render a page much quicker and with less need for redrawing as the DOM is constructed.</p>
<p>I have yet to see the Google AMP Cache in action, but now that the site is live with the tweaked AMP HTML and JS, I will monitor <a href="https://www.google.com/webmasters/tools/">Google Webmaster Tools</a> for AMP errors so Google can start caching the pages. From what I can tell this is an automated process that happens when the pages are crawled, and not something a developer has direct control over (other than to fix errors).</p>
<h2 id="so-what-did-i-change">So what did I change?</h2>
<p>Most of the changes I needed to make were fairly minor. These included:</p>
<ul>
<li>Adding CSS boilerplate code for AMP</li>
<li>Inlining current CSS in amp-custom style tag</li>
<li>Including required AMP JS library. This also included the form, service worker, and analytics extension scripts</li>
<li>Fixing the markup that was output by the <a href="http://rouge.jneen.net/">Rouge code highlighter</a></li>
</ul>
<p>There were a couple of changes that required a lot more work:</p>
<ul>
<li>Converting standard img tags into <a href="https://www.ampproject.org/docs/reference/amp-img.html">amp-img</a> tags</li>
<li>Making sure all images had a width and height attributes</li>
</ul>
<p>The use of amp-img gives a few performance benefits including lazy-loading and srcset support (polyfilled if not native to the browser). AMP-img requires a width and height attribute to be set so it can calculate the correct aspect-ratio of the image. In my case this was very time consuming task, as I had to manually add the width and height attributes for each image used across the blog. If I were to be using a standard CMS, I’d be able to add these attributes very quickly then let the server do all the work!</p>
<p>An example of a basic AMP HTML page is given below. As you can see it is all very standard apart from a couple of custom components from AMP. For more information on the AMP HTML setup <a href="https://www.ampproject.org/docs/get_started/create/basic_markup.html">see here</a>.</p>
<figure class="highlight"><code class="language-html" data-lang="html"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td class="code"><pre><span class="cp">&lt;!doctype html&gt;</span>
<span class="nt">&lt;html</span> <span class="na">amp</span> <span class="na">lang=</span><span class="s">"en"</span><span class="nt">&gt;</span>
<span class="nt">&lt;head&gt;</span>
<span class="nt">&lt;meta</span> <span class="na">charset=</span><span class="s">"utf-8"</span><span class="nt">&gt;</span>
<span class="nt">&lt;title&gt;</span>Hello, AMPs<span class="nt">&lt;/title&gt;</span>
<span class="nt">&lt;link</span> <span class="na">rel=</span><span class="s">"canonical"</span> <span class="na">href=</span><span class="s">"http://example.ampproject.org/article-metadata.html"</span> <span class="nt">/&gt;</span>
<span class="nt">&lt;meta</span> <span class="na">name=</span><span class="s">"viewport"</span> <span class="na">content=</span><span class="s">"width=device-width,minimum-scale=1,initial-scale=1"</span><span class="nt">&gt;</span>
<span class="nt">&lt;style </span><span class="na">amp-boilerplate</span><span class="nt">&gt;body</span><span class="p">{</span><span class="nl">-webkit-animation</span><span class="p">:</span><span class="n">-amp-start</span> <span class="m">8s</span> <span class="n">steps</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="n">end</span><span class="p">)</span> <span class="m">0s</span> <span class="m">1</span> <span class="nb">normal</span> <span class="nb">both</span><span class="p">;</span><span class="nl">-moz-animation</span><span class="p">:</span><span class="n">-amp-start</span> <span class="m">8s</span> <span class="n">steps</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="n">end</span><span class="p">)</span> <span class="m">0s</span> <span class="m">1</span> <span class="nb">normal</span> <span class="nb">both</span><span class="p">;</span><span class="nl">-ms-animation</span><span class="p">:</span><span class="n">-amp-start</span> <span class="m">8s</span> <span class="n">steps</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="n">end</span><span class="p">)</span> <span class="m">0s</span> <span class="m">1</span> <span class="nb">normal</span> <span class="nb">both</span><span class="p">;</span><span class="nl">animation</span><span class="p">:</span><span class="n">-amp-start</span> <span class="m">8s</span> <span class="n">steps</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="n">end</span><span class="p">)</span> <span class="m">0s</span> <span class="m">1</span> <span class="nb">normal</span> <span class="nb">both</span><span class="p">}</span><span class="k">@-webkit-keyframes</span> <span class="n">-amp-start</span><span class="p">{</span><span class="nt">from</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">hidden</span><span class="p">}</span><span class="nt">to</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">visible</span><span class="p">}}</span><span class="k">@-moz-keyframes</span> <span class="n">-amp-start</span><span class="p">{</span><span class="nt">from</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">hidden</span><span class="p">}</span><span class="nt">to</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">visible</span><span class="p">}}</span><span class="k">@-ms-keyframes</span> <span class="n">-amp-start</span><span class="p">{</span><span class="nt">from</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">hidden</span><span class="p">}</span><span class="nt">to</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">visible</span><span class="p">}}</span><span class="k">@-o-keyframes</span> <span class="n">-amp-start</span><span class="p">{</span><span class="nt">from</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">hidden</span><span class="p">}</span><span class="nt">to</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">visible</span><span class="p">}}</span><span class="k">@keyframes</span> <span class="n">-amp-start</span><span class="p">{</span><span class="nt">from</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">hidden</span><span class="p">}</span><span class="nt">to</span><span class="p">{</span><span class="nl">visibility</span><span class="p">:</span><span class="nb">visible</span><span class="p">}}</span><span class="nt">&lt;/style&gt;&lt;noscript&gt;&lt;style </span><span class="na">amp-boilerplate</span><span class="nt">&gt;body</span><span class="p">{</span><span class="nl">-webkit-animation</span><span class="p">:</span><span class="nb">none</span><span class="p">;</span><span class="nl">-moz-animation</span><span class="p">:</span><span class="nb">none</span><span class="p">;</span><span class="nl">-ms-animation</span><span class="p">:</span><span class="nb">none</span><span class="p">;</span><span class="nl">animation</span><span class="p">:</span><span class="nb">none</span><span class="p">}</span><span class="nt">&lt;/style&gt;&lt;/noscript&gt;</span>
<span class="nt">&lt;style </span><span class="na">amp-custom</span><span class="nt">&gt;</span><span class="c">/* Custom CSS goes here */</span><span class="nt">&lt;/style&gt;</span>
<span class="c">&lt;!-- AMP analytics script added here if required --&gt;</span>
<span class="nt">&lt;script </span><span class="na">async</span> <span class="na">src=</span><span class="s">"https://cdn.ampproject.org/v0.js"</span><span class="nt">&gt;&lt;/script&gt;</span>
<span class="c">&lt;!-- Any additional AMP scripts added here --&gt;</span>
<span class="nt">&lt;/head&gt;</span>
<span class="nt">&lt;body&gt;</span>
<span class="nt">&lt;h1&gt;</span>Welcome to the mobile web<span class="nt">&lt;/h1&gt;</span>
<span class="nt">&lt;/body&gt;</span>
<span class="nt">&lt;/html&gt;</span></pre></td></tr></tbody></table></code></figure>
<h2 id="useful-links">Useful links</h2>
<p>Here are a few links I found useful, and are a good starting point if you wish to implement AMP on your own site.</p>
<ul>
<li><a href="https://github.com/ampproject/amphtml">Project GitHub</a></li>
<li><a href="https://validator.ampproject.org/">AMP HTML validator</a></li>
<li><a href="https://chrome.google.com/webstore/detail/amp-validator/nmoffdblmcmgeicmolmhobpoocbbmknc">Chrome validator extension</a></li>
<li><a href="http://stackoverflow.com/questions/tagged/amp-html">Stackoverflow questions</a></li>
</ul>
<h2 id="useful-image-size-shell-script">Useful image size shell script</h2>
<p>As I mentioned earlier in the post I use Jekyll as my static site generator. Because of this I had to add all the height and width attributes manually. Not a simple task when there are over 400 images that need fixing. To speed up the task I put together a very simple shell script. It loops through any images in a given directory, and outputs the width and height of each in the correct format for my Jekyll setup.</p>
<figure class="highlight"><code class="language-shell" data-lang="shell"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="c">#!/bin/bash</span>
<span class="nb">shopt</span> <span class="nt">-s</span> nullglob
<span class="k">for </span>f <span class="k">in</span> <span class="k">*</span>.jpg <span class="k">*</span>.png <span class="k">*</span>.gif
<span class="k">do
</span>identify <span class="nv">$f</span> | awk <span class="s1">'{print $1" "$3}'</span> | awk <span class="s1">'{print $1; split($2,a,"x"); print "width: "a[1]"\nheight: "a[2]"\n"}'</span>
<span class="k">done</span></pre></td></tr></tbody></table></code></figure>
<p>An example of the output of the script to the console is given below. This can easily be copy / pasted into the relevant markdown file.</p>
<figure class="highlight"><code class="language-plaintext" data-lang="plaintext"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="code"><pre>01-initial.gif
width: 767
height: 764
02-rigid.gif
width: 766
height: 764
03-large-panel.gif
width: 766
height: 764
featured.gif
width: 600
height: 250</pre></td></tr></tbody></table></code></figure>
<h2 id="fixing-jekyll-rouge">Fixing Jekyll Rouge</h2>
<p>AMP is very picky about the structure of your HTML. If your pages don’t validate then they won’t be cached by Google, which is what you want for optimum performance on mobile. The main problem I ran into here was Jekyll Rouge was creating HTML that wasn’t AMP valid. A couple of inline style attributes on a table element were enough to invalidate the page.</p>
<p>Looking at the Rouge GitHub page I could see there was already an <a href="https://github.com/jneen/rouge/issues/149">open issue</a>; and it isn’t going to be fixed anytime soon. Luckily there is a way using the Liquid template engine to modify the HTML output before final compilation:</p>
<figure class="highlight"><code class="language-liquid" data-lang="liquid"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre>{ % if _code contains '&lt;pre class="lineno"&gt;' % }
{ % assign _code = _code | replace: '&lt;table style="border-spacing: 0"&gt;', '&lt;table&gt;' % }
{ % assign _code = _code | replace: '&lt;td class="gutter gl" style="text-align: right"&gt;', '&lt;td class="gutter gl"&gt;' % }
{ % endif % }</pre></td></tr></tbody></table></code></figure>
<p>The above code looks for the <code class="highlighter-rouge">pre class="lineno"</code> element, and does a search and replace on the HTML where required. To use, wrap your Liquid highlight include like so:</p>
<figure class="highlight"><code class="language-liquid" data-lang="liquid"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
</pre></td><td class="code"><pre>{ % capture _code % }{ % highlight plaintext linenos=table % }
{ % endhighlight % }{ % endcapture % }{ % include fixlinenos.html % }{ { _code } }</pre></td></tr></tbody></table></code></figure>
<p>And there you have it, valid AMP HTML coming from Rouge. Just remember to add the required CSS back into your standard stylesheet.</p>
<p><strong>Note</strong>: I have added spaces around the liquid tags to stop Jekyll getting confused when compiling. Remove these spaces when you add them to your build.</p>
<h2 id="final-thoughts">Final thoughts</h2>
<p>AMP is fairly simple to implement, although it does come with a strict set of functional restrictions which you will need to work with for your pages to validate. There’s an <a href="https://www.ampproject.org/docs/reference/extended.html">extensive range of components</a> that can be used today, with more being developed and tested all the time. Time will tell on whether the extra development effort is worth it in terms of performance and usability. I’ve defiantly noticed a speed increase of the site on a mobile when loading over a 3G connection (although I haven’t timed it to be 100% sure it isn’t a placebo). I will be monitoring my analytics data closely over the coming weeks, to see if there’s any data that can support a case for overall improvement. But is it the future of the mobile Web? Ask me again in 5 years and I’ll tell you! It is certainly an interesting approach that you should consider investigating if you want to increase your mobile user base.</p>Matt HobbsCould the future of the mobile Web be upon us? Speed up your mobile pages using AMP!