Show Caseshttps://developers.google.com/web/showcase/?utm_source=feed&utm_medium=feed&utm_campaign=showcase_feed
Learn why and how other developers have used the web to create amazing web experiences for their users.19 Nov 2018 00:00:00 GMT19 Nov 2018 00:00:00 GMTHandlebarsJShttps://developers.google.com/web/images/web-fundamentals-icon192x192.pngShow Caseshttps://developers.google.com/web/showcase/?utm_source=feed&utm_medium=feed&utm_campaign=showcase_feed
Nikkei achieves a new level of quality and performance with their multi-page PWA19 Nov 2018 00:00:00 GMThttps://developers.google.com/web/showcase/2018/nikkei?utm_source=feed&utm_medium=feed&utm_campaign=showcase_feed
https://developers.google.com/web/showcase/2018/nikkei?utm_source=feed&utm_medium=feed&utm_campaign=showcase_feed

Nikkei achieves a new level of quality and performance with their multi-page PWA

With a publishing history of more than 140 years, Nikkei
is one of the most authoritative media businesses in Japan. Along with their
print newspaper, they have over 450 million monthly visits to their digital
properties. To provide a better user experience and accelerate their business
on the web, Nikkei successfully launched a Progressive Web App (PWA) -
https://r.nikkei.com - in November 2017. They’re now seeing amazing
results from the new platform.

Business overview

Challenge

Nikkei saw a rapid rise in mobile traffic to their legacy website as
smartphones became the main point of entry to the web for many users.
However, using Lighthouse, an auditing tool that
scans a web page and gives recommendations on how to improve across multiple
categories, they understood that their site wasn’t fully optimized for mobile
across multiple areas and was very slow to load.

Their website was taking ~20 secs to become consistently interactive and
averaged 10 seconds on the Speed Index. Knowing that 53% of mobile users
will abandon an experience if it takes more than 3 seconds to
load, Nikkei wanted to reduce their load time to provide a better experience
and accelerate their business on the web.

The value of speed is indisputable, especially for financial news. We made
speed one of our core metrics, and our customers have appreciated the
change. Taihei Shigemori, Manager, Digital Strategy

The PWA is a multi-page app (MPA) that reduces front-end complexity,
built with Vanilla JavaScript. Five core front-end engineers worked for a
year to achieve this performance.

The Nikkei front-end engineers have proved that great UX brings good
business performance. We’re fully invested in continuing our journey of
bringing a new level of quality to the web. Hiroyuki Higashi,
Product Manager, Nikkei

Solution

Nikkei created and launched a Progressive Web App, using responsive design,
vanilla JavaScript, and a multi-page architecture, they focused on building a
delightful user experience. By adding a service worker, they were able to
provide predictable performance, regardless of the network. This
also ensures that top articles are always available and loaded almost
immediately because they're stored using Cache Storage. They added a web app
manifest, and together with their service worker this allows users to
install the PWA, so it’s easily accessible. And to ensure
performance was entirely within their control, they optimized their
3rd-party JavaScript.

Best practices

Improve loading speed and interactivity by using modern web APIs,
compression, and code optimization practices.

Progressively enhance UX by adding PWA features such as offline support
and Add to Home Screen.

Build performance budgets into performance strategy.

Technical Deep Dive

Speed matters

Speed is more important than ever. As smartphones became the main browsing
device for many users, Nikkei saw a rapid increase of mobile traffic on
their service. But using Lighthouse, they realized
that their legacy website wasn’t fully optimized for mobile, with the Speed
Index averaging 10 seconds, very slow initial load, and a large JavaScript
bundle. It was time for Nikkei to rebuild their website and adapt
web-performance best practices. Here are the results and key performance
optimizations in the new PWA, <r.nikkei.com>.

Leveraging web APIs & best practices to speed loading

Preload key requests

Avoid multiple, costly round trips to any origin

The website needed to load 3rd party resources for tracking, ads and many
other use cases. They used
<link rel=preconnect>
to pre-resolve DNS/TCP/SSL handshake and negotiation for these key 3rd party
origins.

Dynamically prefetch the next page

When they were confident that the user will navigate to a certain page, they
didn’t just wait for the navigation to happen. Nikkei dynamically adds
<link rel=prefetch>
to the <head> and pre-fetches the next page before the user actually clicks
the link. This enables instant page navigation.

Inline Critical-path CSS

Reducing render blocking CSS is one of the best practices of speed loading. The
website inlines all the critical CSS with 0
render blocking stylesheets.
This optimization reduced first meaningful paint by more than 1 second.

Optimize JavaScript bundles

In their previous experience, Nikkei's JavaScript bundles were bloated,
weighing over 300KB in total. Through a rewrite to vanilla JavaScript and
modern bundling optimizations, such as route-based chunking and tree-shaking,
they were able to trim this bloat. They reduced their JavaScript bundle size
by 80%, dropping it to 60KB with RollUp.

Optimizing third-party JavaScript

While it’s not as easy to optimize 3rd party JavaScripts compared to your
own scripts, Nikkei successfully minified and bundled all ad-related scripts,
which are now served from its own content delivery network (CDN). Ad-related
tags usually provide a snippet to initiate and load other required scripts,
which often block the page rendering and also require extra network
turnaround time for each of the scripts downloaded. Nikkei took the following
approach and improved initialization time by 100ms, plus reduced JS size by 30%:

Bundle all the required scripts using a JS bundler (e.g., Webpack)

Async load the bundled script, so that it doesn’t block the page rendering

Attach the calculated ad banner to Shadow DOM (vs iframe)

Progressively load ads on user scroll with Intersection Observer API

Progressively enhancing the website

In addition to these basic optimizations, Nikkei leveraged
Web App Manifest and
service workers to make their
website installable and even work
offline. By using the cache-first
strategy in their service worker, all core resources and top articles are
stored in the Cache Storage and reused even in contingency situations such as
a flaky or offline network, providing consistent, optimized performance.

Hack the Nikkei

A traditional daily newspaper company with a history of 140+ years successfully
accelerated its digitalization through the power of web and PWA. Nikkei’s
front-end engineers proved that great UX delivers strong business performance.
The company will continue its journey of bringing a new level of quality to the web.

Progressive Web App at Voot

TL;DR

Voot.com launches their media Progressive Web App in India. Within days of
implementation, session time per user increased by 39% and daily views per user
by 15%.

Results

80% reduction in page load time

39% increase in session time per user

19% reduction in bounce rate

77% increase in conversion from visitor to video viewer

15% increase in average daily video views per user

About voot.com

Voot is one of India's top video-on-demand products, available as both a native
app and a mobile web app. It offers close to 35,000 hours of premium content
online, including exclusive shows from networks such as Colors and MTV, as well
as Voot Originals and over 8,000 videos for children. Voot is run by Viacom18, a
joint venture between Viacom and Network18 Group.

Challenge

Mobile devices are a primary form of media consumption in India. Such devices
often have limited storage for native apps, making the mobile web a critical
part of the Voot product strategy.

Most mobile Voot users access the Internet via metered 2G and 3G networks. Data
transfer is expensive, and while users become highly committed to a video once
it starts playing, slow load times lead to users giving up before the experience
can even start. To address that challenge and increase mobile web usage, Voot
looked to enhance their site with Progressive Web App features.

Solution

To improve the overall user experience, Voot.com turned their site into a
Progressive Web App, using features such as Add to Homescreen and a service
worker.

While 4G services have recently launched, most users in India
reach the Internet via 2G and 3G networks, with slow and sometimes
expensive data transfer rates. To reduce the data being transferred, Voot.com
optimized their images specifically for mobile, and the
site now dynamically serves either JPEG or WebP images depending upon browser
capabilities.

Voot.com also decreased load times by adding a service worker to preload images
as the user navigates, and to cache images for repeat visits. The site was also
analyzed with Lighthouse to gain performance insights,
which in turn allowed a reduction in JavaScript and CSS overhead and the
implementation of synchronous server calls.

These optimizations resulted in a 63% reduction in first-view page load data
transfer and an 86% reduction in data transfer for returning visitors. Page
loads became 5 times faster for first-time visitors, and almost 7 times faster
for returning users.

To offer a true app-like experience, Voot.com also implemented
add to home screen,
allowing users to launch the page from their home screen like a native app. This
combined with a streamlined login process allows users to quickly and easily
re-engage with the site and watch their favorite shows.

Voot launched its new UI first on mobile web, ahead of desktop web and native
app. Within days of launch, the improvements translated to a 19% reduction in
user abandonment and a 77% increase in conversion from visitor to video viewer.
Users also showed higher levels of engagement, with a 39% increase in average
user session time and a 15% increase in average daily video views per user.
These user engagement metrics show Voot.com performing similarly to the native
app, without requiring the user to install anything.

“We have moved the needle very significantly when it comes to user experience
on the Mobile Web by adopting PWA” says Rajneel Kumar, SVP - Head of Product
and Technology for OTT and Digital Ventures at Viacom18. “All the time and
effort we’ve spent on technology and UI changes as well as optimizations seem
to be showing very positive results. We are going to continue to refine this
further and we are confident that we will continue to see significant consumer
lift.”

George.com is a leading UK clothing brand, part of ASDA
Walmart. After upgrading their site to a Progressive Web App (PWA), the brand
saw a 31 percent increase in mobile conversion.

3.8x - Faster average page load time

2x - Lower bounce rate

31% - Increase in Conversion Rate

20% - More page views per visit

28% - Longer average time on site for visits from Home screen

Challenge

With consumer expectations around the mobile shopping experience at an all-time
high, Asda George realized they needed to revamp an outdated mobile solution and
thereby improve the offer for customers. The team embraced a mobile-first approach,
placing focus on design, speed and functionality to drive mobile conversion.

Solution

The George.com team recognized that to meet this challenge the business had to
enhance the mobile experience by building a Progressive Web App. Working with
Isobar UK to assess the end to end customer journey, the team adopted a scrum
driven, agile approach to the workstream. By incrementally deploying the PWA, the
Asda George team were able to realize the benefits immediately.

Site speed was the most crucial part of the initiative. Following PWA best
practices, in accordance with Google, Asda George reduced page load time for
shoppers by 3.8x times. The business also saw a 31 percent
increase in conversion and 20 percent more page views per visit.

By implementing site wide HTTPS, Asda George now offers a more secure end-to-end
shopping experience for customers, enabling modern browser capabilities, like
Service Worker, and therefore allowing consumers to stay in touch with the brand
whilst offline.

In addition, the brand implemented an “Add to Home Screen” prompt, which resulted
in an increase in customer time on site by 28 percent, truly
creating a native app like experience on the web.

Measuring the Real-world Performance Impact of Service Workers

One of the most significant benefits of service workers (from a performance perspective, at least) is their ability to proactively control the caching of assets. A web application that can cache all of its necessary resources should load substantially faster for returning visitors. But what do these gains actually look like to real users? And how do you even measure this?

The Google I/O web app (IOWA for short) is a progressive web app that leveraged most of the new capabilities offered by service workers to deliver a rich, app-like experience to its users. It also used Google Analytics to capture key performance data and usage patterns from its large and diverse user audience.

This case study explores how IOWA used Google Analytics to answer key performance questions and report on the real-world impact of service workers.

Starting with the questions

Any time you implement analytics in a website or application, it's important to start by identifying the questions you're trying to answer from the data you'll be collecting.

While we had several questions we wanted to answer, for the purposes of this case study, let's focus on two of the more interesting ones.

1. Is service worker caching more performant than the existing HTTP caching mechanisms available in all browsers?

We already expect pages to load faster for returning visitors than for new visitors since browsers can cache requests and serve them instantly on repeat visits.

Service workers offers alternative caching capabilities that give developers fine-grained control over exactly what and how caching is done. In IOWA, we optimized our service worker implementation so that every asset would be cached, so returning visitors could use the app completely offline.

But would this effort be any better than what the browser already does by default? And if so, how much better? [1]

2. How does service worker impact the experience of the site loading?

In other words, how fast does it feel like the site is loading, regardless of the actual load times as measured by traditional page load metrics?

Answering questions about how an experience feels is obviously not an easy task, and no metric is going to perfectly represent such a subjective sentiment. That being said, there are definitely some metrics that are better than others, so choosing the right ones is important.

Choosing the right metric

Avg. Page Load Time is a good metric for answering our first question, but it's not a particularly good metric for answering the second. For one thing the load event doesn't necessarily correspond to the moment when the user can actually interact with the app. Moreover, two apps with the exact same load time may feel like they load much differently. For example, a site with a splash screen or a loading indicator probably feels like it loads much faster than a site that just shows a blank page for several seconds.

In IOWA, we showed a splash screen countdown animation that (in my opinion) very successfully served to entertain the user while the rest of the app loaded in the background. Because of this, tracking how long it takes the splash screen to appear makes a lot more sense as a way to measure perceived load performance. We chose the metric time to first paint to get this value.

Once we decided on the questions we wanted to answer and identified the metrics that would be useful in answering them, it was time to implement Google Analytics and start measuring.

The analytics implementation

The first line in the above code initializes a global ga() function (if it doesn't already exist), and the last line asynchronously downloads the analytics.js library.

The middle part contains these two lines:

ga('create', 'UA-XXXXX-Y', 'auto');
ga('send', 'pageview');

These two commands track what pages are visited by people going to your site, but not much more. If you want to track additional user interactions, you have to do so yourself.

For IOWA, we wanted to track two additional things:

The elapsed time between when the page first starts to load and when pixels appear on the screen.

Whether or not a service worker is controlling the page. With this information we could segment our reports to compare the results with and without service worker.

Capturing time to first paint

Some browsers record the precise time at which the first pixel is painted to the screen, and they make that time available to developers. That value, compared with the navigationStart value exposed via the Navigation Timing API gives us a very accurate accounting of how much time has passed between when the user initially requested the page and when they first saw something.

As I already mentioned, time to first paint is an important metric to measure because it's the first point at which a user experiences the load speed of your site. It's the first impression users get, and a good first impression can positively affect the rest of the user experience. [2]

To get the first paint value in browsers that expose it, we created the getTimeToFirstPaintIfSupported utility function:

Important: the firstpaint values referenced in the above function are part of vendor-specific APIs. They are non-standard and subject to change (or removal) at any time. We chose to implement them in IOWA knowing the site would have a limited lifespan. For most sites, it's best to use other user timing techniques until a firstpaint (or a firstpaint alternative) is standardized.

With this, we could now write another function that sends a non-interaction event with the time to first paint as its value: [3]

After writing both of these functions, our tracking code looks like this:

// Creates the tracker object.
ga('create', 'UA-XXXXX-Y', 'auto');
// Sends a pageview for the initial pageload.
ga('send', 'pageview');
// Sends an event with the time to first paint data.
sendTimeToFirstPaint();

Note that depending on when the above code runs, pixels may or may not have already been painted to the screen. To ensure we always run this code after the first paint occurs, we postponed the call to sendTimeToFirstPaint() until after the load event. In fact, we decided to postpone sending all analytics data until after the page was loaded to ensure those requests wouldn't compete with the loading of other resources.

// Creates the tracker object.
ga('create', 'UA-XXXXX-Y', 'auto');
// Postpones sending any hits until after the page has fully loaded.
// This prevents analytics requests from delaying the loading of the page.
window.addEventListener('load', function() {
// Sends a pageview for the initial pageload.
ga('send', 'pageview');
// Sends an event with the time to first paint data.
sendTimeToFirstPaint();
});

The code above reports firstpaint times to Google Analytics, but that's only half the story. We still needed to track the service worker status; otherwise we wouldn't be able to compare the first paint times of a service worker-controlled page and a non-controlled page.

Determining service worker status

To determine the current status of the service worker, we created a utility function that returns one of three values:

controlled: a service worker is controlling the page. In the case of IOWA that also means all assets have been cached and the page works offline.

supported: the browser supports service worker, but the service worker is not yet controlling the page. This is the expected status for first time visitors.

This function got the service worker status for us; the next step was to associate this status with the data we were sending to Google Analytics.

Tracking custom data with custom dimensions

By default, Google Analytics gives you lots of ways to subdivide your total traffic into groups based on attributes of the user, session, or interaction. These attributes are known as dimensions. Common dimensions web developers care about are things like Browser, Operating System, or Device Category.

The service worker's status is not a dimension Google Analytics provides by default; however, Google Analytics does give you the ability to create your own custom dimensions and define them however you want.

For IOWA, we created a custom dimension called Service Worker Status and set its scope to hit (i.e. per-interaction). [4] Each custom dimension you create in Google Analytics is given a unique index within that property, and in your tracking code you can reference that dimension by its index. For example, if the index of the dimension we just created were 1, we could update our logic as follows to send the firstpaint event to include the service worker status:

ga('send', 'event', {
eventCategory: 'Performance',
eventAction: 'firstpaint',
// Rounds to the nearest millisecond since
// event values in Google Analytics must be integers.
eventValue: Math.round(timeToFirstPaint)
// Sends this as a non-interaction event,
// so it doesn't affect bounce rate.
nonInteraction: true,
// Sets the current service worker status as the value of
// `dimension1` for this event.
dimension1: getServiceWorkerStatus()
});

This works, but it will only associate the service worker's status with this particular event. Since Service Worker Status is something that's potentially useful to know for any interaction, it's best to include it with all data sent to Google Analytics.

To include this information in all hits (e.g. all pageviews, events, etc.) we set the custom dimension value on the tracker object itself, prior to sending any data to Google Analytics.

ga('set', 'dimension1', getServiceWorkerStatus());

Once set, this value gets sent with all subsequent hits for the current pageload. If the user loads the page again later, a new value will likely be returned from the getServiceWorkerStatus() function, and that value will be set on the tracker object.

A quick note on code clarity and readability: since other people looking at this code may not know what dimension1 refers to, it's always best to create a variable that maps meaningful dimension names to the values analytics.js will use.

// Creates a map between custom dimension names and their index.
// This is particularly useful if you define lots of custom dimensions.
var customDimensions = {
SERVICE_WORKER_STATUS: 'dimension1'
};
// Creates the tracker object.
ga('create', 'UA-XXXXX-Y', 'auto');
// Sets the service worker status on the tracker,
// so its value is included in all future hits.
ga('set', customDimensions.SERVICE_WORKER_STATUS, getServiceWorkerStatus());
// Postpones sending any hits until after the page has fully loaded.
// This prevents analytics requests from delaying the loading of the page.
window.addEventListener('load', function() {
// Sends a pageview for the initial pageload.
ga('send', 'pageview');
// Sends an event with the time to first paint data.
sendTimeToFirstPaint();
});

As I mentioned, sending the Service Worker Status dimension with every hit allows us to use it when reporting on any metric. For example, here's what service worker usage looked like for all pageviews during the week of Google I/O:

The results: answering our questions

Once we started collecting data to answer our questions, we could report on that data to see the results. (Note: all Google Analytics data shown here represents actual web traffic to the IOWA site from May 16-22, 2016).

The first question we had was: Is service worker caching more performant than the existing HTTP caching mechanisms available in all browsers?

To answer that question, we created a custom report that looked at the metric Avg. Page Load Times across various dimensions. This metric is well-suited to answer this question because the load event fires only after all initial resources are downloaded. Thus it directly reflects the total load time for all the site's critical resources. [5]

The dimensions we chose were:

Our custom Service Worker Status dimension.

User Type, which indicates whether this is the user's first visit to the site or if they're returning. (Note: a new visitor will not have any resources cached; a returning visitor might.)

Device Category, which lets us compare the results across mobile and desktop.

To control for the possibility that non-service-worker-related factors were skewing our load time results, we limited our query to only include browsers that support service worker.

The chart below shows the average page load times across desktop and mobile for service worker controlled pages and non-controlled pages:

As you can see, visits to our app when controlled by a service worker loaded quite a bit faster than non-controlled visits, even those from returning users who likely had most of the page's resources cached. It's also interesting to notice that, on average, visitors on mobile with a service worker saw faster loads than new desktop visitors.

"…visits to our app when controlled by a service worker loaded quite a bit faster than non-controlled visits…"

You can see more details in the following two tables:

Avg. Page Load Time (Desktop)

Service Worker Status

User Type

Avg. Page Load Time (ms)

Sample Size

Controlled

Returning Visitor

2568

30860

Supported

Returning Visitor

3612

1289

Supported

New Visitor

4664

21991

Avg. Page Load Time (Mobile)

Service Worker Status

User Type

Avg. Page Load Time (ms)

Sample Size

Controlled

Returning Visitor

3760

8162

Supported

Returning Visitor

4843

676

Supported

New Visitor

6158

5779

You may be wondering how it's possible for a returning visitor whose browser supports service worker to ever be in a non-controlled state. There are a few possible explanations for this:

The user left the page on the initial visit before the service worker had a chance to finish initializing.

The user uninstalled the service worker via the developer tools.

Both of these situations are relatively rare. We can see that in the data by looking at the Page Load Sample values in the fourth column. Notice that the middle rows have a much smaller sample than the other two.

Our second question was: How does service worker impact the experience of the site loading?

To answer this question, we created another custom report for the metric Avg. Event Value and filtered the results to only include our firstpaint events. We used the dimensions Device Category and our custom Service Worker Status dimension:

To better visualize how the firstpaint events related to the overall page load times, we created a timeline chart plotting both events for mobile and desktop:

Contrary to what I would have expected, the service worker on mobile had much less impact on time to first paint than it did on overall page load.

"…service worker on mobile had much less impact on time to first paint than it did on overall page load."

To explore why this is the case, we have to dig deeper into the data. Averages can be good for general overviews and broad strokes, but to really get a sense of how these numbers break down across a range of users, we need to look at a distribution of firstpaint times.

Getting the distribution of a metric in Google Analytics

To get the distribution of firstpaint times we need access to the individual results for each event. Unfortunately, Google Analytics doesn't make this easy.

Google Analytics lets us break down a report by whatever dimension we want, but it doesn't let use break down a report by metrics. That isn't to say it's impossible, it just means we had to customize our implementation a bit more to get the desired result.

Since report results can only be broken down by dimensions, we had to set the metric value (in this case firstpaint time) as a custom dimension on the event. To do this we created another custom dimension called Metric Value and updated our firstpaint tracking logic as follows:

The Google Analytics web interface does not currently provide a way to visualize the distribution of arbitrary metric values, but with the help of the Google Analytics Core Reporting API and the Google Charts library we could query for the raw results and then construct a histogram ourselves.

For example, the following API request configuration was used to get a distribution of firstpaint values on desktop with a non-controlled service worker.

This API request returns an array of values that look like this (Note: these are just the first five results). The results are sorted from smallest to largest, so these rows represent the fastest times.

API Response Results (first five rows)

ga:dimension2

ga:totalEvents

4

3

5

2

6

10

7

8

8

10

Here's what these results mean in plain English:

There were 3 events where the firstpaint value was 4 ms

There were 2 events where the firstpaint value was 5 ms

There were 10 events where the firstpaint value was 6 ms

There were 8 events where the firstpaint value was 7 ms

There were 10 events where the firstpaintvalue was 8 ms

etc.

From these results we can extrapolate the firstpaint value for every single event and create a histogram of the distribution. We did this for each of the queries we ran.

Here's what the distribution looked like on desktop with a non-controlled (but supported) service worker:

The median firstpaint time for the above distribution is 912 ms.

The shape of this curve is quite typical of load time distributions. Contrast that with the histogram below which shows the distribution of first paint events for visits in which a service worker was controlling the page.

Notice that when a service worker was controlling the page, many visitors experienced a near-immediate first paint, with a median of 583 ms.

"…when a service worker was controlling the page, many visitors experienced a near-immediate first paint…"

To get a better sense of how these two distributions compare to each other, the next chart shows a merged view of the two. The histogram showing non-controlled service worker visits is overlaid on top of the histogram showing controlled visits, and both of those are overlaid on top of a histogram showing both combined.

One thing I found interesting about these results was that the distribution with a controlled service worker still had a bell-shaped curve after the initial spike. I was expecting a large initial spike and then a gradual trail off, I wasn't expecting a second peak in the curve.

When I looked into what might be causing this, I learned that even though a service worker can be controlling a page, its thread can be inactive. The browser does this to save resources—obviously you don't need every service worker for every site you've ever visited to be active and ready at a moment's notice. This explains the tail of the distribution. For some users, there was a delay while the service worker thread started up.

As you can see from the distribution though, even with this initial delay, browsers with service worker delivered content faster than browsers going through the network.

Here's how things looked on mobile:

While we still had a sizeable increase in near-immediate first paint times, the tail was quite a bit larger and longer. This is likely because, on mobile, starting an idle service worker thread takes longer than it does on desktop. It also explains why the difference between average firstpaint time wasn't as big as I was expecting (discussed above).

"…on mobile, starting an idle service worker thread takes longer than it does on desktop."

Here is the breakdown from these variations of median first paint times on mobile and desktop grouped by service worker status:

Median Time to First Paint (ms)

Service Worker Status

Desktop

Mobile

Controlled

583

1634

Supported (not controlled)

912

1933

While building these distribution visualizations took a bit more time and effort than creating a custom report in Google Analytics, they give us a far better sense of how service workers affect the performance of our site than averages alone.

Other impact from Service Workers

In addition to the performance impact, service workers also impact the user experience in several other ways that are measurable with Google Analytics.

Offline access

Service workers allow users to interact with your site while offline, and while some sort of offline support is probably critical for any progressive web app, determining how critical it is in your case largely depends on how much usage is occurring offline. But how do we measure that?

Sending data to Google Analytics requires an internet connection, but it doesn't require the data to be sent at the exact time the interaction took place. Google Analytics supports sending interaction data after the fact by specifying a time offset (via the qt parameter).

For the past two years IOWA has been using a service worker script that detects failed hits to Google Analytics when the user is offline and replays them later with the qt parameter.

To track whether the user was online or offline, we created a custom dimension called Online and set it to the value of navigator.onLine, we then listened for the online and offline events and updated the dimension accordingly.

And to get a sense for how common it was for a user to be offline while using IOWA, we created a segment that targeted users with at least one offline interaction. Turns out, that was almost 5% of users:

Push notifications

Service workers allow users to opt-in to receiving push notifications. In IOWA, users were notified when a session in their schedule was about to start.

As with any form of notifications, it's important to find the balance between providing value to the user and annoying them. To better understand which is happening, it's important to track whether users are opting-in to receive these notifications, if they're engaging with them when they arrive, and if any users who previously opted-in change their preference and opt-out.

In IOWA, we only sent notifications related to the user's personalized schedule, something only logged-in users could create. This limited the set of users who could receive notifications to logged-in users (tracked via a custom dimension called Signed In) whose browsers supported push notifications (tracked via another custom dimension called Notification Permission).

The following report is based on the metric Users and our Notification Permission custom dimension, segmented by users who signed in at some point and whose browsers support push notifications.

It's great to see that more than half of our signed-in users opted to receive push notifications.

App install banners

If a progress web app meets the criteria and is used frequently by a user, that user may be shown an app install banner, prompting them to add the app to their home screen.

In IOWA, we tracked how often these prompts were shown to the user (and whether they were accepted) with the following code:

Of the users who saw an app install banner, about 10% chose to add it to their home screen.

Possible tracking improvements (for next time)

The analytics data we collected from IOWA this year was invaluable. But hindsight always brings to light holes and opportunities to improve things for the next time around. After finishing this year's analysis, here are two things I wish we'd done differently that readers looking to implement a similar strategy might want to consider:

1. Track more events related to the load experience

We tracked several events that correspond to a technical metric (e.g. HTMLImportsLoaded, WebComponentsReady, etc.), but since so much of the load was done asynchronously, the point at which these events fired didn't necessarily correspond with a particular moment in the overall load experience.

The primary load-related event we didn't track (but wish we had) is the point at which the splash screen disappeared and the user could see the page content.

2. Store the analytics client ID in IndexedDB

By default, analytics.js stores the client ID field in the browser's cookies; unfortunately, service worker scripts cannot access cookies.

This presented a problem for us when we tried to implement notification tracking. We wanted to send an event from the service worker (via the Measurement Protocol) each time a notification was sent to a user, and then track the re-engagement success of that notification if the user clicked on it and arrived back in the app.

While we were able to track the success of notifications in general via the utm_sourcecampaign parameter, we weren't able to tie a particular re-engagement session to a particular user.

What we could have done to work around this limitation was store the client ID via IndexedDB in our tracking code, and then that value would have been accessible to the service worker script.

3. Let the service worker report online/offline status

Inspecting navigator.onLine will let you know if your browser is able to connect to the router or local area network, but it won't necessarily tell if you if the user has real connectivity. And since our offline analytics service worker script simply replayed failed hits (without modifying them, or marking them as failed), we were probably under-reporting our offline usage.

In the future we should track both the status of navigator.onLine as well as whether the hit was replayed by the service worker due to an initial network failure. This will give us a more accurate picture of the true offline usage.

Wrapping up

This case study has shown that using service worker did indeed improve the load performance of the Google I/O webapp across a wide range of browsers, networks, and devices. It has also shown that when you look at a distribution of the load data across a wide range of browsers, networks, and devices, you get much more insight into how this technology handles real-world scenarios, and you discover performance characteristics that you may not have expected.

Here were some of the key takeaways from the IOWA study:

On average, pages loaded quite a bit faster when a service worker was controlling the page than they did without a service worker, for both new and returning visitors.

Visits to pages controlled by a service worker loaded almost instantly for many users.

Service workers, when inactive, took a bit of time to start up. However, an inactive service worker still performed better than no service worker.

The startup time for an inactive service worker was longer on mobile than on desktop.

While the performance gains observed in one particular application are generally useful to report to the larger developer community, it's important to remember that these results are specific to the type of site IOWA is (an event site) and the type of audience IOWA has (mostly developers).

If you're implementing service worker in your application, it's important that you implement your own measurement strategy so you can assess your own performance and prevent future regression. If you do, please share your results so everyone can benefit!

Footnotes

It's not entirely fair to compare the performance of our service worker cache implementation to the performance of our site with HTTP cache alone. Because we were optimizing IOWA for service worker, we didn't spend much time optimizing for HTTP cache. If we had, the results probably would have been different. To learn more about optimizing your site for HTTP cache, read Optimizing Content Efficiently.

Depending on how your site loads its styles and content, it's possible that the browser is able to paint prior to content or styles being available. In such cases, firstpaint may correspond to a blank white screen. If you use firstpaint, it's important to ensure it corresponds to a meaningful point in the loading of your site's resources.

Technically we could send a timing hit (which are non-interaction by default) to capture this information instead of an event. In fact, timing hits were added to Google Analytics specifically to track load metrics like this; however, timing hits get heavily sampled at processing time, and their values cannot be used in segments. Given these current limitations, non-interaction events remain better suited.

Ele.me is the biggest food ordering and delivery company
in mainland China. It serves 260 million registered users from over 200
cities all around China, and has over 1.3 million restaurant listings.
With 99% of its users ordering food on mobile, Ele.me set out to improve its
mobile web experience, making it faster and more reliable in flaky connections,
all while relying on the core technical model of a multi-page app to
accommodate their operational needs.

Loading time decreased by 11.6% across all pre-cached pages

Loading time decreased on average by 6.35% across all pages.

Time-to-consistently-interactive dropped to 4.93 seconds on a 3G network on
first load

After we released the ele.me PWA, our loading times have dropped
significantly, transforming our mobile web experience into one of the
fastest food reservation sites in China.

Spencer Yang, Product Manager of Ele.me PWA

Choosing between multi-page app and single page app

In a multi-page app (MPA), every route that a user navigates to triggers
a full request of the page, along with associated scripts and styles needed,
to the server. This is in contrast to a single-page app (SPA) model, where
every route navigation triggers a fetch just for the content and data
relevant to that route, and the UI is then constructed by Javascript code
running on the client app.

The explosive growth of Ele.me in recent years has led to the growth of
distinct business units within the company, each in charge of running its
micro-service under the main https://ele.me domain. The
Ele.me team concluded that the decoupling of these individual services is
best served by a multi-page app (MPA) model, with each team running and
maintaining its own service.

Applying PRPL to a MPA

The PRPL pattern (Preload
critical resources, Render initial route, Pre-cache remaining routes,
Lazy-load remaining routes) provides web developers with a set of rails to
guide the structure of a PWA, with a particular emphasis on a quick time to
interactivity and maximizing caching to reduce network round trips. While PRPL
has been well-tested on SPAs, it was less clear how one would actually apply it
on a MPA. Ele.me decided to take on the PRPL mindset when thinking about a
rebuild of their MPA as a PWA. To do that, they make sure that when a user
navigates to a page, that they are preloading critical resources for that
page by including <link rel="preload"> as needed, or surfacing those scripts
at a sufficiently shallow level so that the browser’s preloader can do its work
without needing additional hints. They also progressively enhance their PWA by
installing a service worker whenever it is supported by the browser, which they
then use to fetch and pre-cache other top-level navigation routes so that
the user gets a faster loading and rendering experience as they click around the
PWA. Each page in a MPA is its own route, so speeding up the rendering of the
initial route is equivalent to implementing best practices to optimize the
critical rendering path for each route. With these changes, the overall loading
time decreased on average by 6.35% across all pages.

Serving the transition skeleton screens ASAP

Ele.me wanted to apply the idea of
skeleton screens
into the UX, which is a way to ensure that whenever the user taps any button
or link, the page reacts as soon as possible by transitioning the user to that
new page, and then loading in content to that page as the content becomes
available; this is also the key to improving the perceived performance of
the PWA. However, since each page in a MPA is its own initial route, each
navigation requires redoing all the necessary loading, parsing, and evaluation
work every single time.

To work around this, Ele.me built the skeleton screen as an actual UI
component, and then used Vue.js’ Server Side Rendering stack to build and
then pre-render the Vue components to strings before injecting them into
HTML templates. This allows them to render the skeleton screen directly and
achieve a more fluid transition when navigating between pages.

Caching shared resources with service worker

Different routes are loaded as a user browses around the PWA, and it would be a
waste to load these routes from the network over and over again. To tackle this,
Ele.me analyzed the critical routes that users care about most, created a
webpack plugin to collect the dependencies of these critical routes, and then
precached these routes when they install a service worker on the user’s client
browser. These critical routes include the Javascript, CSS, and images that form
the typical UI shell of the PWA.

Routes that are considered important, but not critical, are incrementally cached
by the service worker at runtime instead as the user continues to navigate
through the PWA. This allows Ele.me to serve the PWA to users directly from
cache under all network conditions. The result: loading time decreased by 11.6%
across all pre-cached pages.

Bear 71 and WebVR

TL;DR

WebVR is an API built into browsers that combines
stereo rendering with real-time head tracking, enabling a quick and easy way to
enjoy VR content online. With WebVR, content creators can create immersive VR
content that lives online and runs on a wide range of VR hardware.

About Bear 71

Bear 71 is an interactive documentary produced by the National Film Board
(NFB) of Canada. Originally built in Flash, Bear 71 was released in 2011 to
critical acclaim. The backbone of the experience is a 23 minute track of audio
and video, highlighting the relationship between humans, nature, technology, and
one particular brown bear. The viewer is a voyeur in a world of information
represented as an abstract grid of symbols. Surprisingly, this doesn't hamper
the emotional story within, and it should be experienced first-hand to be
understood.

Challenge

The original Bear 71 was developed in Flash; arguably the best interactive
storytelling medium of the time. A lot of care went into the technology and it
was considered cutting edge. Five years later, the original vision holds up and
the story is still relevant, but the technology behind it needs an update. Any
technology, used creatively, can assist a good story, but virtual reality
deserves special attention. VR has come and gone several times before, but it's
finally arrived in the mainstream. This is a huge opportunity for storytelling.
Stories which traditionally happened in front of you, can now happen around you
in VR.

When Bear 71 was built, Adobe Flash was in its golden years. Each passing year,
Flash's market share shrinks, and with that, previously accessible works are
harder and harder to view. But with Flash's decline has come the maturation of
HTML 5. Furthermore, VR is being taken more seriously than ever as a real
platform for storytelling.

To our eyes and ears, VR is naturally immersive, but VR has several barriers in
its way. At first glance, the variety of headsets should suit many needs and
budgets, and the variety of app stores involved in distribution is more choice.
But users are known to prefer to do as much in a browser as possible. Users
prefer to download and install as little as possible. For users and content
creators alike, the appeal of HTML5 is the fact that rich experiences are just
one URL away. This is where WebVR comes in.

WebVR enables immersive and comfortable VR content in your browser, across a
multitude of software and hardware.

The NFB recognized this opportunity and considered giving Bear 71 a new life in
HTML5 and WebVR. This opportunity wouldn't be without a new challenge common to
many VR projects: How do we achieve beautiful things, at good framerates, in
stereo vision, on various platforms including mobile phones?

Solution

During the start of development in late 2016, WebVR was not yet readily
available. The WebVR API is evolving quickly, and the WebVR standard is
currently being drafted. But that didn't stop the community from patching
together a suitable stopgap: the
WebVR polyfill. It provided
useable head tracking and stereo rendering via the gyroscope and WebGL. This
polyfill, along with Google Cardboard, allowed us to start building and testing
content. When Google Pixel and Daydream View became available, along with the
first real builds of WebVR in Chrome, our content was ready to take advantage of
it. For quick development, we still include and use the polyfill where WebVR is
not yet available. For instance, we do much of our developing and debugging on
Chrome 55's mobile emulator. Also, it was common to develop and debug on
whatever was readily available, including but not limited to HTC One M9, iPhone
5S, Samsung Galaxy and of course Pixel by Google. The versatility of the
polyfill cannot be understated.

Despite the polyfill's utility, some essential WebVR features simply cannot be
emulated. It's worth noting that one of the biggest hurdles that VR faces is
motion sickness. Motion sickness is tied to, among many things, frame rates and
speed and accuracy of head tracking. WebVR has a few essential features like
reprojection and high-speed orientation sensors. That said, maintaining high
frame rates is still the responsibility of the content creators. Since WebVR
requires rendering the scene twice (once for each eye) per frame, optimization
is doubly important. Properly optimizing your WebGL content is outside the scope
of this article, but here are a few key points:

Reduce your draw calls. A great way of doing this is merging elements that
share a material.

Keep your shaders simple. Standard materials and shaders do a great job of
optimizing out unused features, but sometimes writing brutally optimized
shaders by hand is the best way.

Be sure your art style is economical to render. It's better to look like
Super Mario 64 at 60fps than Mario Galaxy at 6 fps.

Treat the user as a solipsistic: find ways to only simulate the world around the
player.

Look for opportunities to cache results of expensive computations in typed
arrays. Memory is often cheaper than processing power.

This is not an exhaustive list, but it covers most of the optimizations we
used to run at good framerates across most devices.

Creating, repurposing and optimizing the content spanned roughly 12 weeks for a
small team of 8 merry contributors. The effort was both intense and painless.
Bear 71 VR has already made limited appearances at 2 film festivals, with
positive feedback. For those of you not able to find it in the festival circuit,
WebVR can bring the experience to you in the comfort of your browser. If you're
considering a new creative project, remember: the Web is a delivery platform for
rich content that we shouldn't take for granted. VR is here to transport us into
new worlds of our invention. And WebVR is here to combine the best of both.

BookMyShow is India’s largest ticketing firm,
with 50+ million monthly visitors. They developed an improved version of their
mobile website using a Progressive Web App (PWA), delivering an 80+% increase
in conversions, which means more users purchasing tickets.

Since launching our PWA, we’ve seen an exponential increase in mobile
conversion rates. The PWA helped us connect with more people on mobile — and
it's mobile users who make up the majority of our overall audience.

– Sahil Tewari, Assistant General Manager, Mobile & Web

BookMyShow's PWA drove an 80%+ increase in their conversion rates.

The size of the PWA is 54x smaller than the Android app and 180x smaller
than the iOS app.

The PWA takes less than 2.94 seconds to load and enables checkout within
30 seconds.

Challenges

BookMyShow has a growing mobile audience. Over 85% of transactions happen
on mobile, and the company’s mobile web traffic recently overtook their desktop
web traffic. Even with this growth in traffic, the company still encountered
high bounce rates because their mobile website's load speed and user experience
weren’t optimal.

Their native app also posed problems as it required heavy data and memory usage
to be effective. “People were using the native app and were happy with it, but
their main concerns were the data usage and the memory it consumes,” says Anish
Tripathi, Vice President of Design. “And if they uninstalled the app and tried
using the mobile browser, it didn’t work the same way.”

Solution

Knowing that speed and efficiency are key to a good ticket-buying experience,
BookMyShow launched a PWA to deliver the best mobile web experience possible
to the majority of their users. A PWA would provide a smooth, seamless movie-
booking experience that would optimize speed and remove data constraints for
existing customers, without sacrificing the user experience.

The shift to a PWA also gave BookMyShow the opportunity to simultaneously
migrate away from their PHP backend stack, which involved redoing some core
development work. They adopted cutting-edge solutions and built a truly
powerful, technically-advanced web app that provides users with fast,
reliable, and engaging experiences just like the native app.

The PWA is optimized for speed. The goal for the app—which took only 10
months to build—was created to enable checkout within 30 seconds. They
were also excited to use the
“add to home screen” feature
to provide a native-app like experience. Their PWA application is only about
440KB — 54 times smaller than the Android app and 180 times smaller
than the iOS app. BookMyShow also took advantage of service workers to deliver
reliable performance on slow or unreliable networks. Data consumption is also
substantially lower, thanks to optimization. When a user asks for a particular
page, only assets required for that page are loaded, conserving data. On 2G
networks, the initial load time is just 4 seconds. Even for personalized
movie suggestions, the PWA takes less than 2.94 seconds in subsequent
loads. All of these changes resulted in an 80+% increase in conversion
rates, a huge difference in BookMyShow’s bottom line.

Housing.com increases conversions and lowers bounce rate by 40% with new PWA

Results

Company

Housing.com is one of India’s top startups. On track to receive 50 million
visits this year, it has become one of the leading online real estate
platforms in India. After building Housing Go, a
Progressive Web App (PWA), they saw a 38% increase in total conversions
across browsers. The new PWA also delivered higher-value users, with
visitors spending 10% longer per session and returning more often.

Challenges

Housing.com knew their mobile users wouldn't tolerate slow load times, which
significantly affect conversions. They measured the impact of users’ average
page-load time and found that even a one-second improvement brought a
significant boost in the conversion rate.

Poor connectivity and the prevalence of low-end devices also hindered
Housing.com’s growth. They developed a native app so that mobile users
could work offline and re-engage. But current and potential customers were
very data-sensitive, because the cost of Internet access was still high, and
many consumers hesitated to use data or space to download the native app.

Solution

Relying on technological advancements and the improved capabilities of modern
web browsers and web applications such as Service Workers, IndexedDB, Add to
Home Screen, and Push Notifications, Housing.com set out to create a PWA that
was fast, efficient, and reliable. The result was a mobile web experience
that is fast, uses less data, and re-engages users in multiple ways. With the
new PWA, Housing.com saw conversion rates rise 38% and the bounce rate fall
by over 40%.

Most Indian users reach the Internet via 2G and 3G networks, so a fast user
experience is essential. To decrease load times, Housing.com added Service
Workers and streamlined their site to help consumers quickly find what
they’re looking for. Users can continue browsing properties and reviewing
previous searches, all while offline. “High-end smartphones are not that
prevalent in our target user base,” says Vivek Jain, Housing.com CPTO.
“With our new PWA, we could get past our users’ hesitance to download
our app in their low-end devices, yet give them a wonderful experience.”

The new strategy proved effective. Page-load performance increased by
over 30%, and average time-per-session increased 10% across all browsers.
“Understanding the network limitations of our users and the importance
of a great user experience led us to build a PWA,” Jain says. “User
experience is something that defines state-of-the-art applications.”

Related resources

Codelabs

Udacity courses

Intro to Progressive Web Apps
In this course you’ll get started working on your very first Progressive
Web App - a web app that can take advantage of many of the features native
applications have enjoyed. You’ll also get more experience in creating a
web app that works offline using Service Workers. Finally, you’ll make your
app installable to the user’s home screen with the Web App Manifest file.

Company

Launched in April 2010, AliExpress is a global retail marketplace targeted at
consumers worldwide. The platform enables consumers from around the world to buy
directly from wholesalers and manufacturers in China through access to products
at competitive prices. AliExpress is a business within the Alibaba Group.

Challenges

Shoppers from over 200 countries and regions have downloaded the AliExpress app
or used its desktop and mobile websites. In order to make a purchase, users are
required to enter their account information each time, which can be unpleasant.
In addition, users sometimes forget their login information, requiring them to
leave the app and reset their passwords. This is especially problematic when
moving cross platform in the middle of a purchase.

Solution

With Smart Lock, AliExpress was able to improve the user experience in their app
and websites: it’s faster for users to login and start the shopping journey,
users do not need to constantly remember their passwords, and users can be
automatically signed in when moving between platforms like Android and Chrome
web.

Smart Lock on Android prefills 3 out of the 4 required onboarding fields (Email,
First Name, Last Name), reducing manual typing errors. 50% of all new users on
the native app are signed up with the hint selector dialog, and 95% of users
have saved their credentials to Smart Lock for seamless return next time.

After implementing Smart Lock on Chrome with the Credential Management API,
there was a 85% drop in failures during sign-in and a 60% decrease in time spent
signing in.

It was easy for AliExpress to implement Smart Lock. It took only one engineer
and about three days to implement and test the API on Android and a similar
amount of effort on Chrome.

“Since implementing Smart Lock, we’ve been continuously impressed by the
improving data and positive user feedback. This is one of our greatest
accomplishments when it comes to optimizing the user experience for our
sign in process. We’ve seen a reduction in user name and password errors.
Smart Lock is a great product.”

Lijun Chen, Director of AliExpress Mobile Team

Learn more about Smart Lock

Users save passwords to Google from Chrome or Android and passwords are made
available across platforms.

Beyond the Rack

TL;DR

Beyond the Rack re-engages users on the
mobile web, increasing revenue per visit by 26% with push notifications.

"Push notifications allowed us to bring one of the most compelling
capabilities from our native app to our mobile site. We see a direct 20%
click through rate from push notifications—having another channel to reach
our users is a game changer."Richard Cohene, VP Marketing, Beyond the Rack

Key Insights

52% of total users are mobile web

26% is the average increase in spend that occurred by members who visited via push

20% click through rate from push notifications

72% more time spent on the site per visit from members who visited via push notifications

About Beyond the Rack

Beyond the Rack is a leading online retailer that runs sale events of designer products
for its 14 million members globally. Until recently, the company primarily depended on
targeted, daily emails to re-engage users. Recent internal research showed that 52% of
users were visiting their site via the mobile web, so they looked for ways to improve
their mobile engagement and sales. At this point Beyond the Rack reached out to their
mobile shopping platform partner Mobify. Working with Mobify, Beyond the Rack implemented
push notifications on their mobile website and saw a 50% increase in repeat visits within
3 months. The company also found push notifications delivered higher value visits, with
members spending 26% more per visit on average.

Timely, relevant notifications

Every day, Beyond the Rack sends emails to their members about new and exclusive sales
events. With limited sale periods—usually 48 hours—the company couldn’t rely solely on
customers checking their email to generate sales. Beyond the Rack’s new push notifications
provided a 20% click through rate which worked perfectly for flash sales alerts. The
notifications also provided Beyond the Rack with another touchpoint to interact with a
highly-engaged and commercially-valuable audience.

High attention users arrive from push notifications

Beyond the Rack members who visited the site via push notifications spent 72% more
time on the site per visit and shopped more often than the average visitor. By creating
timely pushes, Beyond the Rack increased the relevance and excitement of their pushes — and
customers showed their love through higher sales.

The value of web-based push notifications for business

More than 50% of new members discover Beyond the Rack via the mobile web so having an
avenue to re-engage them after this initial interaction in a personalized way makes it more
than just a platform for discovery. Because push messages appear at the front of a customer’s
mobile phone home screen, they prompt greater response and more immediate action.

About Push Notification

Push notifications enable your mobile web users to choose to receive notifications on their
device just like an installed native app. This lets you effectively re-engage them with customized,
compelling content.

More Information

For more information on Push Notifications on the web check out our resources: