Blog

As the web continues to evolve and new browser technologies continue to emerge, the line between native and web development becomes more and more blurred. New APIs are unlocking the ability to code entirely new categories of software in the browser.

Until recently, the ability to interact with digital musical instruments has been limited to native, desktop applications. The Web MIDI API is here to change that.

In this article, we’ll cover the basics of MIDI and the Web MIDI API to see how simple it can be to create a web app that responds to musical input using JavaScript.

What Is MIDI?

MIDI has been around for a long time but has only recently made its debut in the browser. MIDI (Musical Instrument Digital Interface) is a technical standard that was first published in 1983 and created the means for digital instruments, synthesizers, computers, and various audio devices to communicate with each other. MIDI messages relay musical and time-based information back and forth between devices.

“You must unlearn what you have learned!” Meet the brand new episode of SmashingConf San Francisco with smart front-end tricks and UX techniques. Featuring Yiying Lu, Aarron Draplin, Smashing Yoda, and many others. Tickets now on sale. April 17-18.

A typical MIDI setup might consist of a digital piano keyboard which can send messages relaying information such as pitch, velocity (how loudly or softly a note is played), vibrato, volume, panning, modulation, and more to a sound synthesizer which converts that into audible sound. The keyboard could also send its signals to desktop music scoring software or a digital audio workstation (DAW) which could then convert the signals into written notation, save them as a sound file, and more.

MIDI is a fairly versatile protocol, too. In addition to playing and recording music, it has become a standard protocol in stage and theater applications, as well, where it is often used to relay cue information or control lighting equipment.

MIDI lets digital instruments, synthesizers, and software talk to each other and is frequently used in live shows and theatrical productions. Photo by Puk Khantho on Unsplash.

MIDI In The Browser

The WebMIDI API brings all the utility of MIDI to the browser with some pretty simple JavaScript. We only need to learn about a few new methods and objects.

Introduction

First, there’s the navigator.requestMIDIAccess() method. It does exactly what it sounds like—it will request access to any MIDI devices (inputs or outputs) connected to your computer. You can confirm the browser supports the API by checking for the existence of this method.

Second, there’s the MIDIAccess object which contains references to all available inputs (such as piano keyboards) and outputs (such as synthesizers). The requestMIDIAccess() method returns a promise, so we need to establish success and failure callbacks. And if the browser is successful in connecting to your MIDI devices, it will return a MIDIAccess object as an argument to the success callback.

Third, MIDI messages are conveyed back and forth between inputs and outputs with a MIDIMessageEvent object. These messages contain information about the MIDI event such as pitch, velocity (how softly or loudly a note is played), timing, and more. We can start collecting these messages by adding simple callback functions (listeners) to our inputs and outputs.

Going Deeper

Let’s dig in. To send MIDI messages from our MIDI devices to the browser, we’ll start by adding an onmidimessage listener to each input. This callback will be triggered whenever a message is sent by the input device, such as the press of a key on the piano.

The MIDIMessageEvent object we get back contains a lot of information, but what we’re most interested in is the data array. This array typically contains three values (e.g. [144, 72, 64]). The first value tells us what type of command was sent, the second is the note value, and the third is velocity. The command type could be either “note on,” “note off,” controller (such as pitch bend or piano pedal), or some other kind of system exclusive (“sysex”) event unique to that device/manufacturer.

For the purposes of this article, we’ll just focus on properly identifying “note on” and “note off” messages. Here are the basics:

Note values are on a range from 0–127, lowest to highest. For example, the lowest note on an 88-key piano has a value of 21, and the highest note is 108. A “middle C” is 60.

Velocity values are also given on a range from 0–127 (softest to loudest). The softest possible “note on” velocity is 1.

A velocity of 0 is sometimes used in conjunction with a command value of 144 (which typically represents “note on”) to indicate a “note off” message, so it’s helpful to check if the given velocity is 0 as an alternate way of interpreting a “note off” message.

Given this knowledge, we can expand our getMIDIMessage handler example above by intelligently parsing our MIDI messages coming from our inputs and passing them along to additional handler functions.

Browser Compatibility And Polyfill

As of the writing of this article, the Web MIDI API is only available natively in Chrome, Opera, and Android WebView.

The Web MIDI API is only available natively in Chrome, Opera, and Android WebView.

For all other browsers that don’t support it natively, Chris Wilson’s WebMIDIAPIShim library is a polyfill for the Web MIDI API, of which Chris is a co-author. Simply including the shim script on your page will enable everything we’ve covered so far.

This shim also requires Jazz-Soft.net’s Jazz-Plugin to work, unfortunately, which means it’s an OK option for developers who want the flexibility to work in multiple browsers, but an extra barrier to mainstream adoption. Hopefully, within time, other browsers will adopt the Web MIDI API natively.

Is your pattern library up to date today? Alla Kholmatova has just finished a fully fledged book on Design Systems and how to get them right. With common traps, gotchas and the lessons she learned. Hardcover, eBook. Just sayin’.

Making Our Job Easier With WebMIDI.js

We’ve only really scratched the surface of what’s possible with the WebMIDI API. Adding support for additional functionality besides basic “note on” and “note off” messages starts to get much more complex.

If you’re looking for a great JavaScript library to radically simplify your code, check out WebMidi.js by Jean-Philippe Côté on Github. This library does a great job of abstracting all the parsing of MIDIAccess and MIDIMessageEvent objects and lets you listen for specific events and add or remove listeners in a much simpler way.

Real-World Scenario: Building A Breakout Room Controlled By A Piano Keyboard

A few months ago, my wife and I decided to build a “breakout room” experience in our house to entertain our friends and family. We wanted the game to include some kind of special effect to help elevate the experience. Unfortunately, neither of us have mad engineering skills, so building complex locks or special effects with magnets, lasers, or electrical wiring was outside the realm of our expertise. I do, however, know my way around the browser pretty well. And we have a digital piano.

Thus, an idea was born. We decided that the centerpiece of the game would be a series of passcode locks on a computer that players would have to “unlock” by playing certain note sequences on our piano, a la Willy Wonka.

This is a musical lock

Sound cool? Here’s how I did it.

Setup

We’ll begin by requesting WebMIDI access, identifying our keyboard, attaching the appropriate event listeners, and creating a few variables and functions to help us step through the various stages of the game.

Step 1: Press Any Key To Begin

To kick off the game, let’s have the players press any key to begin. This is an easy first step which will clue them into how the game works and also start a countdown timer.

function noteOn(note) { switch(currentStep) { // If the game hasn't started yet. // The first noteOn message we get will run the first sequence case 0: // Run our start up sequence runSequence('gamestart'); // Increment the currentStep so this is only triggered once currentStep++; break; } } function runSequence(sequence) { switch(sequence) { case 'gamestart': // Now we'll start a countdown timer... startTimer(); // code to trigger animations, give a clue for the first lock break; } }

Step 2: Play The Correct Note Sequence

For the first lock, the players must play a particular sequence of notes in the right order. I’ve actually seen this done in a real breakout room, only it was with an acoustic upright piano rigged to a lock box. Let’s re-create the effect with MIDI.

For every “note on” message received, we’ll append the numeric note value to an array and then check to see if that array matches a predefined array of note values.

We’ll assume some clues in the breakout room have told the players which notes to play. For this example, it will be the beginning of the tune to “Amazing Grace” in the key of F major. That note sequence would look like this.

This is the correct sequence of notes that we’ll be listening for as the solution to the first lock.

Step 3: Play The Correct Chord

The next lock requires the players to play a combination of notes at the same time. This is where our “note off” listener comes in. For every “note on” message received, we’ll add that note value to an array; for every “note off” message received, we’ll remove that note value from the array. Therefore, this array will reflect which notes are currently being pressed at any time. Then, it’s a matter of checking that array every time a note value is added to see if it matches a master array with the correct values.

For this clue, we’ll make the correct answer a C7 chord in root position starting on middle C. That looks like this.

These are the four notes that we’ll be listening for as the solution to the second lock.

Now all that’s left to do is to add some additional UI elements and animations and we have ourselves a working game!

Here’s a video of the entire gameplay sequence from start to finish. This is running in Google Chrome. Also shown is a virtual MIDI keyboard to help visualize which notes are currently being played. For a normal breakout room scenario, this can run in full-screen mode and with no other inputs in the room (such as a mouse or computer keyboard) to prevent users from closing the window.

If you don’t have a physical MIDI device laying around and you still want to try it out, there are a number of virtual MIDI keyboard apps out there that will let you use your computer keyboard as a musical input, such as VMPK. Also, if you’d like to further dissect everything that’s going on here, check out the complete prototype on CodePen.

Conclusion

MIDI.org says that “Web MIDI has the potential to be one of the most disruptive music [technologies] in a long time, maybe as disruptive as MIDI was originally back in 1983.” That’s a tall order and some seriously high praise.

I hope this article and sample app has gotten you excited about the potential that this API has to spur the development of new and exciting kinds of browser-based music applications. In the coming years, hopefully, we’ll start to see more online music notation software, digital audio workstations, audio visualizers, instrument tutorials, and more.

If you want to read more about Web MIDI and its capabilities, I recommend the following:

By now, most people working in tech know and feel the deep concerns related to surveillance capitalism fostered and upheld by the tech giants. We understand that the root of the problem lies within the business model of capitalising and monetising user data. Stories of how people are being exploited surface on a daily basis, like the recent story about how Instagram withholds like notifications to certain users, with the purpose of increasing the rate of which they open the app. In the same story, The Globe and Mail describe how former high level employees of Facebook are growing a conscience and tell horrifying stories about how features are meticulously being built to exploit human behavior and make us addicts of social media.

As designers and developers we have an obligation to build experiences that are better than that. This article explains how unethical design happens, and how to do ethical design through a set of best practices. It also helps you understand how you can plant the seed to change the meaning within the company you work for and in the design community, even if you are not part of the management layer. Change starts with a movement!

Ethical Design

Let’s start with the core terminology: According to Merriam Webster, ethics is “the discipline dealing with what is good and bad and with moral duty and obligation.” For the purpose of this article, ethics will be defined as a system of moral principles that defines what is perceived as good and evil. Ethical design is, therefore, design made with the intent to do good, and unethical design is its black hat counterpart.

Is your pattern library up to date today? Alla Kholmatova has just finished a fully fledged book on Design Systems and how to get them right. With common traps, gotchas and the lessons she learned. Hardcover, eBook. Just sayin’.

Ind.ie is a social enterprise striving for justice in the digital age. It is founded by Aral Balkan and Laura Kalbag who defined an “Ethical Hierarchy of Needs” that describe the core of ethical design very well.

The ‘Ethical Hierarchy of Needs’ (licensed under CC BY 4.0) (Source: ind.ie)

As with any pyramid-shaped structure, the layers in the Ethical Hierarchy of Needs rest on the layer below it. If any layer is broken, the layers resting on top of it will collapse. If a design does not support human rights, it is unethical. If it supports human rights but does not respect human effort by being functional, convenient and reliable (and usable!), then it is unethical. If it respects human effort but does not respect human experience by making a better life for the people using it, then it is still unethical.

From a practical viewpoint, this means that products and services which exploit user data, use dark patterns and generally are only out to make money, disregarding its human purpose, are unethical. Let’s look at how unethical design manifests itself in business models and design decisions.

Unethical Design: The Black Hat Of The Business

Surveillance Capitalism

Data-driven design can be used to do good. But more often than not it is used with monetary intent also known as surveillance capitalism.

“When a company like Facebook improves the experience of its products, it’s like the massages we give to Kobe beef: they’re not for the benefit of the cow but to make the cow a better product. In this analogy, you are the cow.”

Surveillance capitalism is unethical by nature because at its core, it takes advantage of rich data to profile people and understand their behavior with the sole purpose of making money. The most chilling thought of all is how data is being used not just to predict and manipulate current behavior, but how it is used to profile our future selves through machine learning, ultimately giving companies the power to impact our future decisions and behavioral patterns.

As Cracked Labs, independent research institute and creative laboratory, states in their report about Data Against People:

“Systems that make decisions about people based on their data produce substantial adverse effects that can massively limit their choices, opportunities, and life-chances”.

This happens on a daily basis to everyone who use Facebook, where the individualized feed is carefully filtered to show the posts most likely to trigger engagement and activity. Pricing is also becoming increasingly individualized because companies are able to use rich data to assess the long-term value of customers, also known as data-driven persuasion.

Data trade and data tracking is big business. According to the report “Corporate Surveillance in Everyday Life”, Oracle provides access to 5 billion (yes, billion!) unique user ID’s (this is confirmed on Oracle’s website). The word “scared” does not cover the emotional state which we should all be in over that fact.

One can only imagine how companies will be able to utilize data to profile which of us are more likely to develop mental health or physical issues, thus putting us in the “no thanks” pile of applications for our future jobs. With this in mind, I fear for the future of my children.

To some the above sounds like something out of a science fiction movie, but it is not far-fetched at all. Unethical companies are exposed daily, from the VPN app who claimed to protect the data of their 24 million users, but sold it to Facebook to the software called Alphonso which, according to an article in The New York Times, is used in more than 250 game apps (some of which are designed for children) to monitor what tv ads people watch, even if the app is not in active use. The list of companies who harvest and use data with deeply unethical purposes goes on and on and on.

Black Hat Design

Nevertheless, data tracking is not the only way that unethical design plays out. Dark patterns fall under unethical design too, as they are black hat design patterns specifically designed to trick us into doing something we don’t necessarily want to do. It may not be considered unethical when a company makes use of the dark pattern called Roach Motel to make it nearly impossible to delete your account (looking at you, Skype). But looking at the motivation of the business side, it is not hard to see the unethical nature of the design, naming the Chinese shoe company who tricked people to swipe by adding a fake strand of hair on their Instagram ad as just one example.

Harry Brignull, one of the originators and driving forces behind Dark Patterns, states that dark patterns work because they take advantage of the human brain’s weaknesses and the way we are hard-wired. That counts as unethical in my book, and we have a moral obligation to the people who design these products to do better.

Businesses who nurture consumerism and manipulate people into buying more stuff are unethical by design. It is common practice in e-commerce, and it takes advantage of a phenomenon called “loss aversion”. Hotels.com is particularly aggressive, sending several notifications within seconds about how many people have booked and are looking at the same room as you.

Hotels.com loves letting us know just how many others are looking at a room to ‘motivate’ us into booking fast.

The requirement of any organization that collects data to do so in a secure manner by design;

Heavy fines for data breaches;

Data must only be collected after explicit consent, and the language used to explain why the data is collected must be plain and simple. In addition, consent must be withdrawable at any time and must be as easy to do as it was giving it (which, by the way, includes Sign up → Delete profile);

“The right to be forgotten,” meaning that people have the right to have their data deleted;

The right for people to get access to their personal data in any organization, alongside information about how this data is processed;

Data portability, meaning that people have the right to get hold of their data in one company and transfer it to another company;

Heavy fines for non-compliance of GDPR.

All in all, May 25, 2018, is a pretty good day for the people, and a sign of a much different future for all the unethically designed organizations out there.

But let’s not kid ourselves and think that change will happen overnight or across all organizations in a heartbeat. A deeper change is needed in order to make that happen over time.

Change Happens Through Change Of Meaning

GDPR is not likely to solve all of our problems. Believing that would be naive. That’s why it will continue to be up to the people within company walls to make a difference. The good news is that making a change is possible even if you’re not part of the management team who decides on the business model.

This type of change towards a more ethical design approach is not claimed to happen overnight. Rather, it is possible to make changes incrementally and foster long-term, organizational change through what Don Norman and Roberto Verganti call Meaning-driven Innovation (read more in their article about “Incremental and Radical Innovation”).

Meaning-driven innovation is a result of people starting to articulate new thoughts that create new dynamics, which ultimately lead to radically new meanings.

Don Norman and Roberto Verganti say that radical innovation only happens when one of the following things occur:

A new enabling technology is available in a reliable, economical form (it took 20 years for touch interfaces to exit the labs and enter the phone and tablet market);

When the meaning of something changes in society — also referred to as meaning-driven innovation.

I think that the increasing awareness, focus, and concern about our privacy in respect to data-driven businesses will in a not so distinct future spark radical innovation and change in society. Ironically, the companies who founded surveillance capitalism have also sparked a change in how we perceive our right to privacy, and we are already starting to see companies and organizations innovate and foster privacy driven solutions.

The meaning of companies who track and surveil us is already changing. We used to not think very hard about, and maybe even find it convenient when ads served on Facebook were based on our browsing history. But with the escalation of surveillance capitalism, I will argue that we are going through a meaning change as an increasing mass of people are finding it not only uncomfortable but directly unacceptable to be spied on in the name of a business.

DuckDuckGo, a search engine that doesn’t track, had an average of 16 million search queries per day in 2017, a sign that privacy is an increasing concern among people when using the web. Also, we are seeing apps devoted to privacy reaching markets, such as Signal, a secure phone and messenger app designed to protect privacy. Furthermore, it is highly likely that GDPR will spark further meaning change related to privacy.

Transition To A Human-Centered Design Approach

Human-Centered Design (HCD) is a philosophy developed by Don Norman (among others). According to the User Experience Professionals Association (UXPA), HCD is defined by “the active involvement of users and a clear understanding of user and task requirements”.

Don Norman and Roberto Verganti conclude in their substantial study that HCD is only suitable for incremental innovation — step by step improvements — because new ideas are not discovered while constantly looking at the existing state of things as it is done through user research.

While this sounds reasonable, I believe that HCD can prove to be the offset to meaning-driven innovation, ultimately leading to a broad meaning change about what people will accept and won’t accept from unethical organizations. The reason why I believe this, is that HCD fosters a deeper sense of empathy than any other experience design method.

Human-Centered Design is a framework as well as a mindset. At its core, working “human-centered” means involving the people you serve early and continuously in the process, i.e. using research to establish the needs of these people, understanding what problems they have, and how your product can help solve these problems.

It falls within the natural sphere of experience designers to work human-centered, but what do you do if your job is in design and development, and you are constantly occupied by sprint reviews and daily tasks?

While working with a remote development team, I learned that the developers didn’t have any contact with the people who used the product. This often led to heated discussions where statements like “I think…”, “From a technical perspective…”, and “I feel…” were the main arguments.

The biggest problem with basing decisions on what you think and feel, or what is easiest from a technical perspective, is that it doesn’t involve the people you are serving. The people who your product or service is put in the world to solve problems for. That’s where HCD comes in.

UX designers and researchers typically conduct research, document the insights and bring them forward in a refined state to the design- and product team in the shape of personas, user needs descriptions, user flows, journeys, and so on. And that’s all well and good. The problem is, however, that the distance between the organisation and people you serve remains large, because no one except the UX designers talked to them or have seen them use the product. So they keep going back to “I feel…”, “I think…” and “From a technical perspective…”

To help establish empathy towards the people you serve, there are a couple of very impactful things designers and developers — and the rest of the organization — can do (and can ask for from the UX team).

Involve all team members in watching videos from user testing sessions.Actually going through the pains and delights of the people who use your products (or prototypes, depending on what you’re testing) is worth every second. It cannot be stressed enough how important it is to watch other people interact with and comment on the stuff you’re building (and no, your team doesn’t count as “people” here!).

If this is not part of the routine in your company, ask for it to be included. Certainly, the vast majority of UX designers who would not be thrilled to organize and facilitate such sessions. You are guaranteed to go through pain, agony, frustration, happiness and get multiple eye openers, and it will all serve as the stepping stone towards growing a human-centered mindset.

Ask for actual, living portraits of the people you serve.This includes photos and video from contextual studies (link), stories from their daily life and stories about them. Getting a deeper sense of the people on the other side of the product you develop creates instant empathy and makes it a lot harder to design things that knowingly are bad for them.

Insist on continuous testing.It cannot be emphasized enough how crucial testing is in HCD. This includes early proof of concept tests, prototype tests, and usability tests. A side bonus of early and continuous involvement from the people who are meant to use the product is that it saves money in the long run. The earlier you realize a bad call or error, the cheaper it is to fix.

A lot has been said about the nuclear alarm that was triggered by mistake in Hawaii on January 13. However, it’s pretty safe to say that early and continuous testing would have helped prevent it.

A poorly designed screen has been named the cause of a false ballistic missile alert in Hawaii. (Source: Honolulu Civil Beat)

Always ask “why?”To start a change of meaning in an organization or community, the first important step is to start asking “why.” Ask why something is being done unethically; ask why you are told to make a black hat feature; question the current state of things.

Ask on what grounds design decisions are being made. If it’s because of what the CEO or someone else thinks, and it has no root in insights from the people you are serving, ask for that validation. Meaning change grows through small steps.

Ethical Design Best Practices

Alongside establishing a human-centered design tradition in the organization, it is also important to make use of the best practices of ethical design. People who do so are part of taking the lead and showing the rest of the organization how things can be done in a more ethical way, all of which will add to the incremental meaning change. Just as dark patterns fall under unethical design, we have White Hat design patterns that can be utilized to ensure ethical design, some of which you can learn about in the following.

Use Data To Improve The Human Experience

Despite numerous companies using data for unethical purposes, such as increasing consumption and traffic, data can, in fact, be used to actually improve the human experience.

This is the case on the ind.ie forum where setting up an account is suggested as a way to customize your experience by remembering what you’ve read.

ind.ie‘s forum highlights the benefits of setting up an account — one that won’t harvest your data and use it unethically.

In a current project in which I am involved in making an app for students to more easily access their Learning Management System, we are sorting the student’s individual courses by “last visited”; we know through research that the students most often revisit a small number of pages that relate to the courses they are currently enrolled in. This customization is not designed to change behavior or nudge them into using parts of the app they didn’t intend to. Instead, it is designed to make their experience faster and more efficient.

In this early wireframe from a current project, it it shown how we used knowledge about current usage of the product to increase efficiency in the app by sorting the student’s courses by ‘last visited’.

Another positive example is the American pharmacy Walgreens, who sends out reminders when it’s time to refill on things like vitamins. This is an extremely helpful feature that solves a problem a lot of us have.

Data can help inform research initiatives to understand how you’re not tackling the problems people have when interacting with your product.

Skyscanner, a travel search engine, noticed through their data that their newly launched design didn’t go down well for the people who used their service to fly out of Amsterdam. That data helped inform a research initiative that ultimately lead to a customized solution for people flying to and from Amsterdam that broke down the barriers the new design had initially built (here’s the background story).

Advertising Without Tracking

Advertising is not necessarily unethical. Advertising based on granular data is. It is careless (if not stupid) to rely on a single platform for the majority of a brand’s marketing efforts, especially considering that said platform owns and controls the data the brand uses as the foundation for its ad targeting.

Facebook, Instagram, and Google care just about as little about their advertisers as they do about the people who they see as “users,” i.e. they can make any change they want to, disregarding any consequences it might have for people or businesses using their platform. For example, there was a time when they started blocking fake accounts en masse, which hurt numerous companies whose social media admins had set up fake accounts to administer business pages because they (understandably) didn’t want to use their private accounts for this purpose. This is standard procedure from Facebook who only allows one profile per person (likely because allowing several would contaminate their data tracking).

The platform is always the weakest link in a marketing strategy because it is a third party beyond the control of companies. Thinking back to the 90’s when I worked in marketing at a regional newspaper, granular data tracking was not an option. When a store had placed an ad in the paper about an event, they would simply monitor how many people showed up, and compare that to their expectations to determine the success rate.

While “the good old days” were certainly not good in all aspects, the thought of not basing advertising on granular user data tracking is an appealing thought. John Gruber’s blog, Daring Fireball is an example of a site that doesn’t allow ad tracking. Instead, John Gruber encourages advertisers to add a custom link to their ads enabling them to monitor the click rate on their end.

As John Gruber rightfully states:

“If you pay (say) Facebook for an ad, why in the world would you, the advertiser, trust Facebook’s numbers for how the ad performed?”

Another great example worth highlighting is Goodwings, a hotel booking site that donates half of their commission to charity. They can do so because of a close collaboration with a large number of NGO’s which means they spend very little on traditional marketing.

Goodwings is a hotel search engine that donates half of their commission to charity.

And if you think Google Analytics is your only option for collecting meaningful data, think again: Matamo is an open source tool that is installed directly on your own server. It guarantees that data is not shared with advertising agencies such as Google.

Always, Always Prioritize Usability

Surveillance capitalism and data tracking are heavily talked about problems related to ethical design these days. But we must not forget the importance of complying with the best practices of usability. Without it, design is unethical, as a lack of usability almost always entails the use of dark patterns. A good place to go to read more about core usability is Nielsen Norman Group.

Back in the early days, Jakob Nielsen defined 5 core components of usability:

Learnability,

Efficiency,

Memorability

Errors, and

Satisfaction.

To ensure a usable product, it’s crucial that these five components are front and center in the design and development process.

While it may not be the most interesting 404 page around, Apple’s version focuses on error prevention by offering us a forward path when we hit a lost page.

Don’t Ask For More Than You Need

As with many other aspects in life, asking for more than you need results in exploitation. It’s common for e-commerce sites to ask for tons of information when people sign up for an account or buy a product. But if someone is buying a digital product (such as a book), there really is no need to ask for anything other than their email address.

Signal is a great example. Private at its core, it doesn’t ask for anything more than the information that is absolutely necessary for people to start using the app right away.

Be Transparent

Norwegian.com has perhaps one of the best airfare booking systems in the world. Not only is it convenient to use but they also offer full transparency in relation to their optional service fees, something that is often hidden away. Anyone who lives outside of an Amazon storage country knows how hard it is to find the actual delivery price of their location.

Conclusion

The movement towards a more ethical future has begun. Change doesn’t happen radically short term unless it’s built into the core of the business model. But that doesn’t mean we cannot change the current state for the better. We can do so through incremental change; one step at a time. By working human-centered, by asking why, and by using best practices for ethical design. That’s our obligation as the ones who build products so deeply ingrained in people’s lives. What we do changes and shapes lives for better or for worse. I choose better.

Learn More

There are a lot of valuable resources available for anyone interested in ethical design. Here are a few to get you started:

Cracked Labs is an independent research institute and a creative laboratory based in Vienna, Austria. It investigates the socio-cultural impacts of information technology and develops social innovations in the field of digital culture. They offer in-depth reports about most things related to privacy.

Service Worker is probably one of the most misrepresented technologies we currently have. When I hear people talking about it, the topic almost always revolves around serving an app when a user is offline. However, Service Worker can do so much more than that, and every week I come across new articles that show how powerful the technology really is.

This month, for example, we can learn how to use Service Worker for cross-tab messaging and to load off requests into the background with the Background Sync API. I think the toolset we now have in our browsers already allows us to build great experiences regardless of the network state. Now it’s up to us to make the experiences so great that users truly love them. And that’s probably the hardest part.

Getting the process just right ain’t an easy task. That’s why we’ve set up ‘this-is-how-I-work’-sessions — with smart cookies sharing what works really well for them. A part of the Smashing Membership, of course.

General

Ed Ellson examined Chrome’s Background Sync API and the retry strategy it uses to perform a request. By allowing synchronization in the background after a first attempt has failed, the API helps us improve the browsing experience for users who go offline or are on unstable connections.

Security

With GraphQL you can query exactly what you want whenever you want. This is amazing for working with an API but also has complex security implications. Instead of asking for legitimate, useful data, a malicious actor could submit an expensive, nested query to overload your server, database, network, or all of these. To prevent this from happening, Max Stoiber shows us how we can secure the GraphQL API in our projects.

Privacy

WebKit is introducing the Storage Access API. The new API targets one of the major issues with Safari’s Intelligent Tracking Protection (ITP): Identifying users who are logged in to a first-party service but view content of it embedded on a third party (YouTube videos on a blog, for example). The Storage Access API allows third-party embeds to request access to their first-party cookies when the user interacts with them. A good solution to protect user privacy by default and allow exceptions on request.

A year after Facebook’s announcement to broadly use Cache-Control: Immutable, Paul Calvano examined how widespread its usage is on the web — apart from the few big players. Interesting research and it’s still sad to see that this useful performance tool is used so little. At Colloq, we use it quite a lot, which saves us a lot of traffic and load on our servers and enables us to serve a lot of pages nearly instantly to recurring users.

JavaScript

Jad Joubran shares how to run fetch in a Web Worker to offload it from the main thread into its own. This could be a useful experiment for tasks where expensive requests are triggered, and maybe even at regular intervals.

CSS

Jens Oliver Meiert’s new article “We write CSS like we did in the 90s, and yes, it’s silly” is a thought-provoking piece that shows that despite more tooling and conventions, we fail to improve how we write CSS. This is not about writing CSS in Javascript or a class naming convention but more about how our tools work and how we try to optimize things while keeping the bigger picture intact.

Preethi Sam shares various tricks on how to create knockout text effects with pure CSS. Interesting to see how many different techniques we have nowadays to create such effects.

Accessibility

Heydon Pickering wrote about building inclusive notifications, not only from a technical perspective but the user experience point of view. This resource gives useful advice for designing notifications, what content to include, and how and when to present them to users.

Work & Life

This week I read an article by Alex Duloz, and his words still stick with me: “When we develop a new application, when we post content on the Internet, whatever we do that people will have access to, we should consider just for a minute if our contribution adds up to the level of dumbness kids/teenagers are exposed to. If it does, we should refrain from going live.” The truth is, most of us, including me, don’t consider this before posting on the Internet. We create funny things, share funny pictures and try to get fame with silly posts. But in reality, we shape society with this. Let’s try to provide more useful resources and make the consumption of this more enjoyable so young people can profit from our knowledge and not only view things we think are funny. “We should always consider how teenagers will use what we release.”

There’s an important article on how unhappiness has grown in America’s population since around the year 2000. It reveals that while income inequality might play a role, the more important aspect is that young people who use a lot of digital media are unhappier than those who use it only up to an hour a day. Interestingly, people who don’t use digital media at all, are unhappy, too, so the outcome of this could be that we should try to use digital media only moderately — at least in our private lives. I bet it’ll make a big difference.

Each application is a unique challenge to produce in its own right, but even more so when you consider that we have to deploy most projects in manydifferentlanguages. Our content has to work not only on the BBC News and Sports websites but on their equivalent apps on iOS and Android, as well as on third-party sites which consume BBC content.

Now consider that there is an increasing array of new platforms such as AMP, Facebook Instant Articles, and Apple News. Each platform has its own limitations and proprietary publishing mechanism. Creating interactive content that works across all of these environments is a real challenge. I’m going to describe how we’ve approached the problem at the BBC.

Example: Canonical vs. AMP

This is all a bit theoretical until you see it in action, so let’s delve straight into an example.

Our Visual Journalism content begins with the Donald Trump illustration, and is inside an iframe

This is the canonical version of the article, i.e., the default version, which you’ll get if you navigate to the article from the homepage.

Nope, we can’t do any magic tricks, but we have articles, books and webinars featuring techniques we all can use to improve our work. Smashing Members get a seasoned selection of magic front-end tricks — e.g. live designing sessions and perf audits, too. Just sayin’! 😉

This looks like the same content as the normal article, but is pulling in a different iframe designed specifically for AMP

While the canonical and AMP versions look the same, they are actually two different endpoints with different behavior:

The canonical version scrolls you to your chosen country when you submit the form.

The AMP version doesn’t scroll you, as you cannot scroll the parent page from within an AMP iframe.

The AMP version shows a cropped iframe with a ‘Show More’ button, depending on viewport size and scroll position. This is a feature of AMP.

As well as the canonical and AMP versions of this article, this project was also shipped to the News App, which is yet another platform with its own intricacies and limitations. So how do we do support all of these platforms?

Tooling Is Key

We don’t build our content from scratch. We have a Yeoman-based scaffold which uses Node to generate a boilerplate project with a single command.

Out of the box, this works pretty well for compiling for one platform but we need to support multiple platforms. Let’s delve into some code.

Embed vs. Standalone

In Visual Journalism, we sometimes output our content inside an iframe so that it can be a self-contained “embed” in an article, unaffected by the global scripting and styling. An example of this is the Donald Trump interactive embedded in the canonical example earlier in this article.

On the other hand, sometimes we output our content as raw HTML. We only do this when we have control over the whole page or if we require really responsive scroll interaction. Let’s call these our “embed” and “standalone” outputs respectively.

Each platform has its own wrapper implementation conforming to a common interface of wrapper methods. The wrapper wraps itself around our content and handles the complexity for us.

Simple ‘scrollTo’ implementation by the standalone wrapper

The standalone wrapper’s implementation of the scrollTo function is very simple, passing our argument directly to window.scrollTo under the hood.

Now let’s look at a separate wrapper implementing the same functionality for the iframe:

Advanced ‘scrollTo’ implementation by the embed wrapper

The “embed” wrapper takes the same argument as in the “standalone” example but manipulates the value so that the iframe offset is taken into account. Without this addition, we would have scrolled our user somewhere completely unintended.

The Wrapper Pattern

Using wrappers results in code that is cleaner, more readable and consistent between projects. It also allows for micro-optimisations over time, as we make incremental improvements to the wrappers to make their methods more performant and accessible. Your project can, therefore, benefit from the experience of many developers.

So, what does a wrapper look like?

Wrapper Structure

Each wrapper essentially comprises three things: a Handlebars template, wrapper JS file, and a SASS file denoting wrapper-specific styling. Additionally, there are build tasks which hook into events exposed by the underlying scaffolding so that each wrapper is responsible for its own pre-compilation and cleanup.

Finally, js/wrapper.js contains the iframed implementation of the wrapper API, detailed below. It is shipped separately to the project, rather than compiled in with the application code — we flag wrapper as a global in our Webpack build process. This means that though we deliver our application to multiple platforms, we only compile the code once.

Wrapper API

The wrapper API abstracts a number of key browser interactions. Here are the most important ones:

scrollTo(int)

Scrolls to the given position in the active window. The wrapper will normalise the provided integer before triggering the scroll so that the host page is scrolled to the correct position.

getScrollPosition: int

Returns the user’s current (normalized) scroll position. In the case of the iframe, this means that the scroll position passed to your application is actually negative until the iframe is at the top of the viewport. This is super useful and lets us do things such as animate a component only when it comes into view.

onScroll(callback)

Provides a hook into the scroll event. In the standalone wrapper, this is essentially hooking into the native scroll event. In the embed wrapper, there will be a slight delay in receiving the scroll event since it is passed via postMessage.

viewport: {height: int, width: int}

A method to retrieve the viewport height and width (since this is implemented very differently when queried from within an iframe).

toggleFullScreen

In standalone mode, we hide the BBC menu and footer from view and set a position: fixed on our content. In the News App, we do nothing at all — the content is already full screen. The complicated one is the iframe, which relies on applying styles both inside and outside the iframe, coordinated via postMessage.

markPageAsLoaded

Tell the wrapper your content has loaded. This is crucial for our content to work in the News App, which will not attempt to display our content to the user until we explicitly tell the app our content is ready. It also removes the loading spinner on the web versions of our content.

List Of Wrappers

In the future, we envisage creating additional wrappers for large platforms such as Facebook Instant Articles and Apple News. We have created six wrappers to date:

Standalone Wrapper

The version of our content that should go in standalone pages. Comes bundled with BBC branding.

Embed Wrapper

The iframed version of our content, which is safe to sit inside articles or to syndicate to non-BBC sites, since we retain control over the content.

AMP Wrapper

This is the endpoint which is pulled in as an amp-iframe into AMP pages.

News App Wrapper

Our content must make calls to a proprietary bbcvisualjournalism:// protocol.

Core Wrapper

Contains only the HTML — none of our project’s CSS or JavaScript.

JSON Wrapper

A JSON representation of our content, for sharing across BBC products.

Wiring Wrappers Up To The Platforms

For our content to appear on the BBC site, we provide journalists with a namespaced path:

The journalist puts this “include path” into the CMS, which saves the article structure into the database. All products and services sit downstream of this publishing mechanism. Each platform is responsible for choosing the flavor of content it wants and requesting that content from a proxy server.

/include/newsspec/15996-trump-tracker/english/index/amp/include/newsspec/15996-trump-tracker/english/index/core/include/newsspec/15996-trump-tracker/english/index/envelope ...and so on

This solution can scale to incorporate more platform types as they arise.

Abstraction Is Hard

Building a “write once, deploy anywhere” architecture sounds quite idealistic, and it is. For the wrapper architecture to work, we have to be very strict on working within the abstraction. This means we have to fight the temptation to “do this hacky thing to make it work in [insert platform name here].” We want our content to be completely unaware of the environment it is shipped in — but this is easier said than done.

Features Of The Platform Are Hard To Configure Abstractly

Before our abstraction approach, we had complete control over every aspect of our output, including, for example, the markup of our iframe. If we needed to tweak anything on a per-project basis, such as add a title attribute to the iframe for accessibility reasons, we could just edit the markup.

Now that the wrapper markup exists in isolation from the project, the only way of configuring it would be to expose a hook in the scaffold itself. We can do this relatively easily for cross-platform features, but exposing hooks for specific platforms breaks the abstraction. We don’t really want to expose an ‘iframe title’ configuration option that’s only used by the one wrapper.

We could name the property more generically, e.g. title, and then use this value as the iframe title attribute. However, it starts to become difficult to keep track of what is used where, and we risk abstracting our configuration to the point of no longer understanding it. By and large, we try to keep our config as lean as possible, only setting properties that have global use.

Component Behaviour Can Be Complex

On the web, our sharetools module spits out social network share buttons that are individually clickable and open a pre-populated share message in a new window.

In the News App, we don’t want to share through the mobile web. If the user has the relevant application installed (e.g. Twitter), we want to share in the app itself. Ideally, we want to present the user with the native iOS/Android share menu, then let them choose their share option before we open the app for them with a pre-populated share message. We can trigger the native share menu from the app by making a call to the proprietary bbcvisualjournalism:// protocol.

Native share menu on Android

However, this screen will be triggered whether you tap ‘Twitter’ or ‘Facebook’ in the ‘Share your results’ section, so the user ends up having to make their choice twice; the first time inside our content, and a second time on the native popup.

This is a strange user journey, so we want to remove the individual share icons from the News app and show a generic share button instead. We are able to do this by explicitly checking which wrapper is in use before we render the component.

Generic share button used in the News App

Building the wrapper abstraction layer works well for projects as a whole, but when your choice of wrapper affects changes at the component level, it’s very difficult to retain a clean abstraction. In this case, we’ve lost a little abstraction, and we have some messy forking logic in our code. Thankfully, these cases are few and far between.

How Do We Handle Missing Features?

Keeping abstraction is all well and good. Our code tells the wrapper what it wants the platform to do, e.g. “go full screen.” But what if the platform we’re shipping to can’t actually go full-screen?

The wrapper will try its best not to break altogether, but ultimately you need a design which gracefully falls back to a working solution whether or not the method succeeds. We have to design defensively.

Let’s say we have a results section containing some bar charts. We often like to keep the bar chart values at zero until the charts are scrolled into view, at which point we trigger the bars animating to their correct width.

Bar chart showing values relevant to my area

But if we have no mechanism to hook into the scroll position — as is the case in our AMP wrapper — then the bars would forever remain at zero, which is a thoroughly misleading experience.

How the bar chart could look if scrolling events aren’t forwarded

We are increasingly trying to adopt more of a progressive enhancement approach in our designs. For example, we could provide a button which will be visible for all platforms by default, but which gets hidden if the wrapper supports scrolling. That way, if the scroll fails to trigger the animation, the user can still trigger the animation manually.

We could display a fallback button instead, which triggers the animation on click.

Plans For The Future

We hope to develop new wrappers for platforms such as Apple News and Facebook Instant Articles, as well as to offer all new platforms a ‘core’ version of our content out of the box.

We also hope to get better at progressive enhancement; succeeding in this field means developing defensively. You can never assume all platforms now and in the future will support a given interaction, but a well-designed project should be able to get its core message across without falling at the first technical hurdle.

Working within the confines of the wrapper is a bit of a paradigm shift, and feels like a bit of a halfway house in terms of the long-term solution. But until the industry matures onto a cross-platform standard, publishers will be forced to roll out their own solutions, or use tooling such as Distro for platform-to-platform conversion, or else ignore entire sections of their audience altogether.

It’s early days for us, but so far we’ve had great success in using the wrapper pattern to build our content once and deliver it to the myriad of platforms our audiences are now using.

Have you ever used a plugin and wished it did something a bit differently? Perhaps you needed something unique that was beyond the scope of the settings page of the plugin.

I have personally encountered this, and I’m betting you have, too. If you’re a WordPress plugin developer, most likely some of your users have also encountered this while using your plugin.

Here’s a typical scenario: You’ve finally found that plugin that does everything you need — except for one tiny important thing. There is no setting or option to enable that tiny thing, so you browse the documentation and find that you can’t do anything about it. You request the feature in the WordPress plugin’s support forum — but no dice. In the end, you uninstall it and continue your search.

Imagine if you were the developer of this plugin. What would you do if a user asked for some particular functionality?

The ideal thing would be to implement it. But if the feature was for a very special use case, then adding it would be impractical. It wouldn’t be good to have a plugin setting that only 0.1% of your users would have a use for.

You’d only want to implement features that affect the majority of your users. In reality, 80% of users use 20% of the features (the 80/20 rule). So, make sure that any new feature is highly requested, and that 80% of your users would benefit from it, before implementing it. If you created a setting for every feature that is requested, then your plugin would become complicated and bloated — and nobody wants that.

Your best bet is to make the plugin extensible, code-wise, so that other people can enhance or modify it for their own needs.

In this article, you’ll learn about why making your plugin extensible is a good idea. I’ll also share a few tips of how I’ve learned to do this.

Getting the process just right ain’t an easy task. That’s why we’ve set up ‘this-is-how-I-work’-sessions — with smart cookies sharing what works really well for them. A part of the Smashing Membership, of course.

What Makes A Plugin Extensible?

In a nutshell, an extensible plugin means that it adheres to the “O” part of the SOLID principles of object-oriented programming — namely, the open/closed principle.

If you’re unfamiliar with the open/closed principle, it basically means that other people shouldn’t have to edit your code in order to modify something.

Applying this principle to a WordPress plugin, it would mean that a plugin is extensible if it has provisions in it that enable other people to modify its behavior. It’s just like how WordPress allows people to “hook” into different areas of WordPress, but at the level of the plugin.

A Typical Example Of A Plugin

Let’s see how we can create an extensible plugin, starting with a sample plugin that isn’t.

Suppose we have a plugin that generates a sidebar widget that displays the titles of the three latest posts. At the heart of the plugin is a function that simply wraps the titles of those three posts in list tags:

While this code works and gets the job done, it isn’t quite extensible.

Why? Because the function is set in its own ways, there’s no way to change its behavior without modifying the code directly.

What if a user wanted to display more than three posts, or perhaps include links with the posts’ titles? There’s no way to do that with the code above. The user is stuck with how the plugin works and can nothing to change it.

Including A Hundred Settings Isn’t The Answer

There are a number of ways to enhance the plugin above to allow users to customize it.

One such way would be to add a lot of options in the settings, but even that might not satisfy all of the possibilities users would want from the plugin.

What if the user wanted to do any of the following (scenarios we’ll revisit later):

display WooCommerce products or posts from a particular category;

display the items in a carousel provided by another plugin, instead of as a simple list;

perform a custom database query, and then use those query’s posts in the list.

If we added a hundred settings to our widget, then we would be able to cover the use cases above. But what if one of these scenarios changes, and now the user wants to display only WooCommerce products that are currently in stock? The widget would need even more settings to accommodate this. Pretty soon, we’d have a gazillion settings.

Also, a plugin with a huge list of settings isn’t exactly user-friendly. Steer away from this route if possible.

So, how would we go about solving this problem? We’d make the plugin extensible.

Adding Our Own Hooks To Make It Extensible

By studying the plugin’s code above, we see a few operations that the main function performs:

It gets posts using get_posts.

It generates a list of post titles.

It returns the generated list.

If other people were to modify this plugin’s behavior, their work would mostly likely involve these three operations. To make our plugin extensible, we would have to add hooks around these to open them up for other developers.

In general, these are good areas to add hooks to a plugin:

around and within the major processes,

when building output HTML,

for altering post or database queries,

before returning values from a function.

A Typical Example Of An Extensible Plugin

Taking these rules of thumb, we can add the following filters to make our plugin extensible:

add myplugin_get_posts_args for modifying the arguments of get_posts,

add myplugin_get_posts for overriding the results of get_posts,

add myplugin_list_item for customizing the generation of a list entry,

add myplugin_get_some_post_titles for overriding the returned generated list.

I’m adding a lot of hooks here, which might seem impractical because the sample code is quite simple and small, but it illustrates my point: By adding just four hooks, other developers can now customize the plugin’s behavior in all sorts of ways.

Namespacing And Context For Hooks

Before proceeding, note two important things about the hooks we’ve implemented:

We’re namespacing the hooks with myplugin_. This ensures that the hook’s name doesn’t conflict with some other plugin’s hook. This is just good practice, because if another hook with the same name is called, it could lead to unwanted effects.

We’re also passing a reference to $ args in all of the hooks for context. I do this so that if others use this filter to change something in the flow of the code, they can use that $ args parameter as a reference to get an idea of why the hook was called, so that they can perform their adjustments accordingly.

The Effects Of Our Hooks

Remember the unique scenarios I talked about earlier? Let’s revisit those and see how our hooks have made them possible:

If the user wants to display WooCommerce products or posts from a particular category, then either they can use the filter myplugin_get_posts_args to add their own arguments for when the plugin queries posts, or they can use myplugin_get_posts to completely override the posts with their own list.

If the user wants to display the items in a carousel provided by another plugin, instead of as a simple list, then they can override the entire output of the function with myplugin_get_some_post_titles, and instead output a carousel from there.

If the user wants to perform a custom database query and then use that query’s posts in the list, then, similar to the first scenario, they can use myplugin_get_posts to use their own database query and change the post array.

Much better!

A Quick Example Of How To Use Our Filters

Developers can use add_filter to hook into our filters above (or use add_action for actions).

Taking our first scenario above, a developer can just do the following to display WooCommerce products using the myplugin_get_posts_args filter that we created:

We Can Also Use Action Hooks

Aside from using apply_filters, we can also use do_action to make our code extensible. The difference between the two is that the first allows others to change a variable, while the latter allows others to execute additional functionality in various parts of our code.

When using actions, we’re essentially exposing the plugin’s flow to other developers and letting them perform other things in tandem.

It might not be useful in our example (because we are only displaying a shortcode), but it would be helpful in others. For example, given an extensible backup plugin, we could create a plugin that also uploads the backup file to a third-party service such as Dropbox.

“Great! But Why Should I Care About Making My Plugin Extensible?”

Well, if you’re still not sold on the idea, here are a few thoughts on why allowing other people to modify your plugin’s behavior is a good idea.

It Opens Up the Plugin to More Customization Possibilities

Everyone has different needs. And there’s a big chance your plugin won’t satisfy all of them, nor can you anticipate them. Opening up your plugin to allow for modifications to key areas of your plugin’s behavior can do wonders.

It Allows People to Introduce Modifications Without Touching the Plugin’s Code

Other developers won’t be forced to change your plugin’s files directly. This is a huge benefit because directly modifying a plugin’s file is generally bad practice. If the plugin gets updated, then all of your modifications will be wiped.

If we add our own hooks for other people to use, then the plugin’s modifications can be put in an external location — say, in another plugin. Done this way, the original plugin won’t be touched at all, and it can be freely updated without breaking anything, and all of the modifications in the other plugin would remain intact.

Conclusion

Extensible plugins are really awesome and give us room for a lot of customization possibilities. If you make your plugin extensible, your users and other developers will love you for it.

Take a look at plugins such as WooCommerce, Easy Digital Downloads and ACF. These plugins are extensible, and you can easily tell because numerous other plugins in WordPress’ plugins directory add functionality to them. They also provide a wide array of action and filter hooks that modify various aspects of the plugins. The rules of thumb I’ve enumerated above have come up in my study of them.

Here are a few takeaways to make your plugin extensible:

Follow the open/closed principle. Other people shouldn’t have to edit your code in order to modify something.

To make your plugin extensible, add hooks in these places:

around and within major processes,

when building the output HTML,

for altering post or database queries,

before returning values from a function.

Namespace your hooks’ names with the name of your plugin to prevent naming conflicts.

Try passing other variables that are related to the hook, so that other people get some context of what’s happening in the hook.

Don’t forget to document your plugin’s hooks, so that other people can learn of them.

Further Reading

Here are some resources if you want to learn more about extending plugins:

It’s quite popular these days, and dare I say a damn fine idea, to build sites with components. Rather than building out entire pages one by one, we build a system of components (think: a search form, an article card, a menu, a footer) and then piece together the site with those components.

JavaScript frameworks like React and Vue emphasize this idea heavily. But even if you don’t use any client-side JavaScript at all to build a site, it doesn’t mean you have to give up on the idea of building with components! By using an HTML preprocessor, we can build a static site and still get all the benefits of abstracting our site and its content into re-usable components.

Static sites are all the rage these days, and rightfully so, as they are fast, secure, and inexpensive to host. Even Smashing Magazine is a static site, believe it or not!

Let’s take a walk through a site I built recently using this technique. I used CodePen Projects to build it, which offers Nunjucks as a preprocessor, which was perfectly up for the job.

A Four-Page Site With A Consistent Header, Navigation, And Footer

This is a microsite. It doesn’t need a full-blown CMS to handle hundreds of pages. It doesn’t need JavaScript to handle interactivity. But it does need a handful of pages that all share the same layout.

Consistent header and footer across all pages

HTML alone doesn’t have a good solution for this. What we need are imports. Languages like PHP make this simple with things like <?php include "header.php"; ?>, but static file hosts don’t run PHP (on purpose) and HTML alone is no help. Fortunately, we can preprocess includes with Nunjucks.

Importing components is possible in languages like PHP

It makes perfect sense here to create a layout, including chunks of HTML representing the header, navigation, and footer. Nunjucks templating has the concept of blocks, which allow us to slot in content into that spot when we use the layout.

Notice the files that are included are named like _file.njk. That’s not entirely necessary. It could be header.html or icons.svg, but they are named like this because 1) files that start with underscores are a bit of-of a standard way of saying they are a partial. In CodePen Projects, it means they won’t try to be compiled alone. 2) By naming it .njk, we could use more Nunjucks stuff in there if we want to.

None of these bits have anything special in them at all. They are just little bits of HTML intended to be used on each of our four pages.

That will buy us a fully functional home page! Nice! Each of the four pages can do the same exact thing, but putting different content in the block, and we have ourselves a little four-page site that is easy to manage.

The index.njk file gets compiled into index.html

For the record, I’m not sure I’d call these little chunks we re-using components. We’re just being efficient and breaking up a layout into chunks. I think of a component more like a re-usable chunk that accepts data and outputs a unique version of itself with that data. We’ll get to that.

Making Active Navigation

Now that we’ve repeated an identical chunk of HTML on four pages, is it possible to apply unique CSS to individual navigation items to identify the current page? We could with JavaScript and looking at window.location and such, but we can do this without JavaScript. The trick is putting a class on the <body> unique to each page and using that in the CSS.

In our _layout.njk we have the body output a class name as a variable:

<body class="{{ body_class }}">

Then before we call that layout on an indivdiual page, we set that variable:

Oh and those icons? Those are just individual .svg files I put in a folder and included like

{% include "../icons/cloud.svg" %}

And that allows me to style them like:

svg { fill: white; }

Assuming the SVG elements inside have no fill attributes already on them.

Authoring Content In Markdown

The homepage of my microsite has a big chunk of content on it. I could certainly write and maintain that in HTML itself, but sometimes it’s nice to leave that type of thing to Markdown. Markdown feels cleaner to write and perhaps a bit easier to look at when it’s lots of copy.

This is very easy in CodePen Projects. I made a file that ends in .md, which will automatically be processed into HTML, then included that in the index.njk file.

Building Actual Components

Let’s consider components to be repeatable modules that as passed in data to create themselves. In frameworks like Vue, you’d be working with single file components that are isolated bits of templated HTML, scoped CSS, and component-specific JavaScript. That’s super cool, but our microsite doesn’t need anything that fancy.

We need to create some “cards” based on a simple template, so we can build things like this:

Creating repeatable components with templates

Building a repeatable component like that in Nunjucks involves using what they call Macros. Macros are deliciously simple. They are like as if HTML had functions!

The whole idea here is to separate data and markup. This gives us some pretty clear, and tangible benefits:

If we need to make a change to the HTML, we can change it in the macro and it gets changed everywhere that uses that macro.

The data isn’t tangled up in markup

The data could come from anywhere! We code the data right into calls to the macros as we’ve done above. Or we could reference some JSON data and loop over it. I’m sure you could even imagine a setup in which that JSON data comes from a sort of headless CMS, build process, serverless function, cron job, or whatever.

Now we have these repeatable cards that combine data and markup, just what we need:

HTML is controlled in the macro, while data can come from anywhere

Make As Many Components As You Like

You can take this idea and run with it. For example, imagine how Bootstrap is essentially a bunch of CSS that you follow HTML patterns in which to use. You could make each of those patterns a macro and call them as needed, essentially componentizing the framework.

You can nest components if you like, embracing a sort of atomic design philosophy. Nunjucks offers logic as well, meaning you can create conditional components and variations just by passing in different data.

In the simple site I made, I made a different macro for the ideas section of the site because it involved slightly different data and a slightly different card design.

It’s possible to create as many components as you want

A Quick Case Against Static Sites

I might argue that most sites benefit from a component-based architecture, but only some sites are appropriate for being static. I work on plenty of sites in which having back-end languages is appropriate and useful.

One of my sites, CSS-Tricks, has things like a user login with a somewhat complex permissions system: forums, comments, eCommerce. While none of those things totally halt the idea of working staticly, I’m often glad I have a database and back-end languages to work with. It helps me build what I need and keeps things under one roof.

Go Forth And Embrace The Static Life!

Remember that one of the benefits of building in the way we did in this article is that the end result is just a bunch of static files. Easy to host, fast, and secure. Yet, we didn’t have to give up working in a developer-friendly way. This site will be easy to update and add to in the future.

CodePen Projects handles all the Nunjucks stuff we talked about here, and also things like Sass processing and image hosting, which I took advantage of for the site. You could replicate the same with, say, a Gulp or Grunt-based build process locally. Here’s a boilerplate project like that you could spin up.

Back in the 2000s, it was commonplace to see websites that were built using Flash. By viewing the source of a website, you’d often see very little HTML and an embedded SWF file. This meant a few things. First, the browser didn’t natively support Flash, so you had to download the Flash plugin. Browsers found it difficult to go into the SWF to read content. Amongst other things, this had a detrimental affect on SEO and accessibility.

In 2007, the iPhone was released. It didn’t support Flash. In 2015, Google moved all its YouTube videos to HTML5. In July 2017, Adobe officially announced it would stop working on Flash by 2020. People used Flash because it could do things that HTML, CSS, and JavaScript couldn’t do at the time. It’s incredible to see how far web standards have come (and what’s coming).

We can do a lot today that was previously only possible with Flash. Not only that, but we can do it in a way that’s far more accessible and performant. I’ll go over some of the groundbreaking things Flash could do and how we can go about doing them today.

Disclaimer: I love Flash, and it will always have a place in my heart, but for me at least, its time has passed. Just in case you’re wondering: there are still so many interfaces and engines running in Flash, especially for games, and this article addresses some of the issues that are very relevant there.

Nope, we can’t do any magic tricks, but we have articles, books and webinars featuring techniques we all can use to improve our work. Smashing Members get a seasoned selection of magic front-end tricks — e.g. live designing sessions and perf audits, too. Just sayin’! 😉

Video

One of the great things Flash heralded was video, offering basic support as early as 2002. It wasn’t until 2009 that the <video> tag was introduced in Chrome, Safari, and Firefox. Furthermore, Internet Explorer (IE) 8 didn’t support the <video> tag, and it wasn’t until 2011, when IE 9 was released, that it got support.

Now, we can simply write <video src="filename.mp4" />, although it is important to be aware of different video formats across browsers, the most popular being MP4, Ogg and WebM. Taking things a step further, it’s possible not only to support the <video> tag, but also offer fallbacks and helpful alternatives:

Interaction And Gaming

Another thing Flash excelled at was interaction and gaming. The popular website Miniclip was founded in 2001 and hosted a wide range of Flash games. In 2008, it was valued at over £900 million and is still going today.

JUST A REFLEKTOR

JUST A REFLEKTOR is an interactive music video that rivals and even surpasses the capabilities of Flash. With the use of various web technologies, it’s now possible to interact with the video using a mobile device, as well as at one point using your webcam so that you actually appeared in the music video yourself!

The website Just A Reflektor makes great use of modern web technologies to create an interactive music video.

Cube Slam

There’s some fantastic web-based Chrome experiments online today, such as Cube Slam. Cube Slam is a game that makes use of WebRTC (an open web technology), letting you video chat and play a game within the browser. Although Flash was heavily used for video chat, it came with a number of drawbacks compared to WebRTC: It relied on the Flash plugin, it required a media server, and it had various security implications and poor performance.

Cube Slam is a web based Chrome Experiment that allows video chat whilst playing a game.

HTML5 Game Engines

There are a number of HTML5 and JavaScript game engines. This next example makes use of canvas and WebGL. WebGL (Web Graphics Library) is an API built in JavaScript that allows interactive 2D and 3D graphics within the <canvas> tag.

As mentioned in Good Boy Digital’s own post regarding the project (the creators of the example):

“Star Wars Arcade really pushes the boundaries of what’s possible with HTML5 and WebGL technologies. This allows for the creation of a single build that works seamlessly on both desktop and mobile browsers without having to download an app; the advantage of this being able to offer an ‘app like’ experience on all devices so anyone can enjoy it, instantly. No passwords, no App Stores, just hit the URL and play!”

I especially love this bit: “Just hit the URL and play!” One of my earliest “Wow” memories of the web was having my own website in 1999 and being able to type that URL into any computer connected to the web and view it. It seemed absolutely incredible to me that this was actually possible (and continues to amaze me to this day!).

Browser Support

One of the advantages of building something—especially a game, due to the extra complexity—in Flash that is still relevant today is browser support. Browser support is generally pretty good these days, and Can I Use can help us to quickly find out about the state of browser support for a particular specification. However, there are still discrepancies that could cause issues. So, if you’re OK with only supporting browsers that are installed with the Flash plugin you’re working with, then you’re likely not to encounter any cross-browser issues.

Typography

Flash was originally designed as an animation tool. As such, it had various limitations with typography.

Flash had a pixel-grid system. If typography was laid on the grid at X:100.3 :100.7 and, thus, out of alignment to the pixel grid, it would look blurred.

As a result, I found that pixel fonts were useful because they sat on the grid and remained crisp. Another limitation here would be if you used an 8-pixel font but set it to 10 pixels, it would go out of alignment with the grid and, again, be blurred.

Thankfully, today in HTML and CSS, we have a host of tools to help us. We can set fonts as an absolute unit in px (pixels) or, more common these days, use ems and rems to aid with responsive web design (I’ll be covering more on this later).

Another issue with Flash and typography was fonts. If a font wasn’t available on the device, a fallback font would be provided. To circumvent this in Flash, you could embed the font in the .swf file. By doing this, though, you added to the file size and, thus, the time that the SWF would take to download and appear.

That being said, what was possible with Flash was Scalable Inman Flash Replacement (sIFR). sIFR allowed HTML text to be replaced with Flash. Before this, in order to use custom fonts, we used images. However, using images didn’t allow for selectable text, and it meant you had to create images manually. Moving on from sIFR, developers came up with Cufón. Cufón avoided the use of Flash by using an SVG and VML version of a font. It was faster than sIFR and didn’t require the Flash plugin; but, again, with this technique, it wasn’t possible to select text.

Today, we have the CSS @font-face rule and a host of standard web fonts available:

In Chrome and Firefox (and, hopefully soon, Safari), we have font-display in CSS. If you’re using a custom font, by default the browser will wait to get the custom font. If it can’t get the custom font, it will use a backup font (the speed varies among browsers, but it is usually 3 seconds). This is known as a flash of invisible text (or FOIT). To improve this scenario, we can use the following:

@font-face { font-display: swap; }

By using swap, we’ll see the text immediately using the backup font. When the custom font is loaded, the browser will swap the backup for it. This way, the user gets to read the content as soon as it’s available.

Animation

One of the things that Flash did very well was tweening. Tweening is used to animate elements. In Flash, you could create an element in a keyframe, duplicate that keyframe along the timeline, and then add a tween.

With HTML and CSS, we can apply the same animation using @keyframes, transform and animation.

It’s also possible to debug potential performance issues that may arise when dealing with animation. In Chrome Developer Tools, there’s a “Performance” tab. By clicking this, then the “Record” circle icon, we can see a range of useful information. This technique greatly helped me when I built Mind’s Annual Report 2012-13, particularly the section of the website that has a map with animated circles showing the locations of Mind shops. Initially, the map section was loaded at the start, which caused repaint issues. Using the “Performance” tab, I was able to identify and update this, so the map only started animating when it was in view.

Vector Graphics

The web has benefitted and still does benefit enormously from careful consideration of file size. Back in the early 2000s, the web was mostly viewed on desktop computers, with slow dial-up modems. A simple image could take seconds or even minutes to load. To help with this, Flash made heavy use of vector graphics. Using vector graphics, where appropriate, instead of JPEG or GIF images, significantly reduced file size and thus load more quickly over the web.

Over the past few years, and particularly thanks to Sara Soueidan, scalable vector graphics (SVGs) have become more and more widespread on the web. SVG is an XML-based markup that enables us to create vector graphics for the web. It works extremely well with animation and I’ve had the pleasure of building some websites that make use of this: the Mind report website (previously mentioned) and How Clean Is England? which Sara mentioned on Twitter! Thanks Sara!

Mind’s Annual Report website made use of SVGs and animation to create a fun way to display their statistics for the year. The How Clean Is England? website was heavily based on illustration. SVGs and CSS animations helped to make the illustrations look crisp and keep file sizes to a minimum.

Responsive Web Design

One of the main pitfalls of building a website in Flash today is the lack of media queries. Today, mobile and tablet usage has surpassed that of desktop. In order to create the best experience, we must create a website that is accessible on all of these devices. On many devices, Flash will simply not load at all, and even if it did, it would most likely breach the viewport’s width or would scale and be unusable.

Using media queries, we can create a layout that responds to the content. Here’s an example:

ActionScript vs. JavaScript

ActionScript is used in Flash and, thus, has the same pitfall of SWF files mentioned earlier, in that it requires the Flash plugin. JavaScript, on the other hand, is readily available in all modern browsers.

Let’s look at an example of setting a variable in both and their differences:

var x:Number = 42;

var x = 42;

With ActionScript, we declare that the variable is a number. If the variable is assigned anything else, it will get an error. JavaScript is loosely typed, which means we could assign the variable as something else, such as a string:

var x = '42';

In JavaScript, if we wanted to check that it is a number, we could use typeof(x);, and this would output “number”. Another option would be to create a function and use isNaN to detect whether it is “not a number”:

function isNumber(value) { if (isNaN(value)) { return value + ' is not a number.'; } return value + ' is a number.'; } console.log(isNumber(42)); // "42 is a number." console.log(isNumber('forty two')); // "forty two is not a number."

Collaboration

With HTML, CSS, and JavaScript (and many other coding languages), Git and GitHub make collaboration extremely easy. For example, if I wanted to edit the HTML of Smashing Magazine’s “Author Template,” via GitHub, I could click the “Fork” button. This would create a version of the files (also known as the repository) under my own name. I could then make any amendments I like and submit a pull request. This would give the owner at Smashing Magazine the ability to review my pull request and accept or reject it. Once accepted, the code would go in the main repository.

There are a number of great reasons for working in this way: You always have a backup of your work; you can revert to previous versions of your work, and collaboration becomes very easy. Someone could be working on one section of the website, or on the CSS or JavaScript, and when each team member has finished, you could review the changes and pull them in as required.

If we tried the same with Flash, it would be a lot more difficult having to save and send an .fla file each time. If multiple people were to work on the same .fla, things could get very confusing. With HTML, CSS and JavaScript, it’s’ possible to do a “diff” on the code, which allows us to compare and review the code. We can even select certain code chunks, bring them in, or comment on them for further review and work.

Conclusion

Flash was one of the reasons I started building websites. It pioneered in a lot of areas, and this led to people creating amazing things with it. Over the years, it’s pushed the web forward a great deal. Adobe’s official announcement of dropping support of Flash, though, does raise concerns. It would be a massive shame if millions of websites using Flash were lost. There’s a petition to open source Flash and Shockwave. I do hope we don’t lose it forever. We’ve had some great — and weird — times. I’ll leave you with this classic example of the “weird” to which I refer:

Over the last few years, I ran several usability studies with participants with various disabilities. I thought it would help others if I shared some of my experiences.

In this article, I provide lessons learned or tips to consider in planning and executing usability testing with participants with disabilities. The lessons learned are divided into general that can apply to all types of disabilities; and lessons learned for three specific disability categories: visual, motor, and cognitive. These tips will help you regardless where you work: If you work with an established user research team where usability testing is part of your design process or if you work on your own with limited resources but want to improve the quality of the user research by expanding the diversity of participants.

Background

Several of our clients from a state government agency to several fortune 500 companies came to us at the User Experience Center (UXC) for help with their websites. They wanted to make sure users with disabilities could access their site and accomplish their goals.

There are many different kinds of disabilities, however, there is a general agreement to categorize people with disability into four general categories: visual, auditory, motor (also referred to as “physical”), and cognitive. There are different conditions and much variability within each category, e.g., within visual disabilities, color blindness, low vision, and blindness. There is also a distinction as to when a disability is contracted, e.g., a person who was born blind as opposed to one who lost vision later on in life.

Getting the process just right ain’t an easy task. That’s why we’ve set up ‘this-is-how-I-work’-sessions — with smart cookies sharing what works really well for them. A part of the Smashing Membership, of course.

Furthermore, as we age or encounter unique situations (such as multi-tasking), we may have a similar experience to people we think of as disabled. Therefore, disabilities should be thought of as a spectrum of abilities that should be accounted for during the design of all user interfaces and experiences.

Typically, in order to ensure that disabled people can use their digital products and services, companies aim for compliance with accessibility guidelines such as the Web Content Accessibility Guidelines (WCAG 2.0). While this is critical, it is also important to have users with disabilities try to accomplish real tasks on the site in usability testing. There may be gaps in the overall user experience…

Think about the typical doors found in buildings. How many times have you tried to open a door one way and realized they actually open the other, for example, push instead of pull. Technically the door is accessible, but it is usable?

Example of doors that are technically accessible, but not usable (handles give impression you should push but must pull to open). (Large preview)

Lesson Learned

In most ways, usability testing with this segment of the population is no different than testing with anyone else. However, there are several areas you need to pay just a bit more attention to so your sessions run smoothly. The lessons or tips are broken down into general ones that can apply to all participants and specific tips for various disability types such as visual, motor, and cognitive.

General Lessons Learned

1. Ensure a baseline level of accessibility before usability testing

Ensure a baseline level of accessibility before usability testing: Planning usability testing, especially recruiting participants can take time both for the project team and the recruited participants.

Two good examples of basic accessibility issues that should be addressed prior to usability testing are:

Missing alternative (alt) text.Usability testing can be used to see if the alt text used is appropriate and makes sense to participants, but if all the participants are doing is confirming that the alt text is missing then this is not a good use of their time.

2. Focus the recruiting strategy

If you work with an external recruiter ask them if they have experience recruiting people with disabilities; some do. If you are recruiting internally (without an external recruiter), you may need to reach out to organizations that have access to people with disabilities. For example, if you need to recruit participants with visually disabilities in the United States, you should contact a local chapter of the National Federation of the Blind (https://nfb.org/state-and-local-organizations) or a local training center such as the Carroll Center for the Blind in Massachusetts (http://carroll.org/). If you use social media to advertise your study, a good approach is to use the hashtag #a11y (stands for accessibility — there are 11 letters between the “a” and “y”) in your post.

3. Bring their own equipment/assistive technology

Allow and encourage participants to bring their own equipment such as their own laptop, especially if they use assistive technology. This way, you can truly see how people customize and use assistive technology.

4. Have a backup plan for assistive technology

As stated above in #3. It is best if participants can bring their own equipment. However, it is always wise to plan for the worst, for example, if a participant does not bring their equipment or if there is a technical problem such as you can’t connect their equipment to your Wi-Fi network. In the case of visually impaired participants, install assistive technology (AT) such as screen reader software they will be bringing in on a backup PC. For many of the AT software packages, you can get a free trail that should cover you for the usability testing period. This has saved us several times. Even though the configuration was different than what the participants had, we were able to run the session. Participants were quickly able to go into the settings and make some adjustments (e.g., increase the speech rate) and get started with the session.

5. Allow additional time

Provide additional time in-between sessions. Typically we like to reserve 30 minutes between participants. However, when participants plan to bring in their own equipment additional time may be required for setting up and resolving any issues that may arise. When testing with individuals with disabilities, we schedule an hour between sessions, so we have extra time for setting up assistive technology and testing it.

6. Confirm participant needs

Either with the recruiting screener or via email or telephone, confirm what equipment participants will bring in and need to be supplied beforehand. In our lab, we can connect external laptops (that in this case, were outfitted with special accessibility software and settings) to our 1Beyond system via an HDMI cable. In a recent study, all of our participants’ laptops had HDMI ports. However, we forgot to check this beforehand. This is an example of a small but important thing to check to prevent show-stopping issues at the time of the test.

7. Consider additional cost

Depending on the disability type transportation to the usability testing location may add additional burden or cost. Consider the cost of transportation in the incentive amount. If feasible, consider providing an extra $ 25-$ 40 in your incentive amount so participants can take a taxi/Uber/Lyft, etc. to and from your location. Depending on access to public transportation and taxi/ride-sharing rates in your area the amount may vary. Our participants came to the UXC in different ways — some more reliable and timely than others.

8. Revise directions

Check the directions you provide for accessibility. Make sure they include an accessible path into your building. Test them out beforehand. Do you need to provide additional signage? If so, ensure all signs are clear, concise, and use plain-language directions.

9. Review the emergency evacuation plan

Review the plan in the event of a fire or other emergency. Map out the emergency evacuation plan in advance.

10. Consider logistics

Consider remote usability testing as an option. One of the benefits of bringing individuals with disabilities into the lab for usability testing is observing first-hand participants’ use of the product or website in question. However, the logistics of getting to your location may be just too much for participants. If it’s possible to test remotely (we typically do this through Zoom or GoToMeeting), it should be considered. This poses the additional challenge of making sure your process for capturing the remote session is compatible with all of the participant’s assistive technology, as well as accessible itself. Troubleshooting remotely is never fun and could be more difficult with this segment of the population.

11. Hearing impaired participants

Some participants may have a hearing impairment where the position of the moderator and participant is critical for adequate communication. In the case of hearing-impaired participants, it is important to get their attention before talking to them and also to take turns when engaging in conversation.

Lessons Learned For Participants With Visual Disabilities

Participants with visual disabilities range from people who are blind and use screen readers such as JAWS, to people that need to the text or the screen to be enlarged using software such as ZoomText or relying on the native screen enlargement in the browser. People that are color-blind also fall into this category.

For any documents needed prior to the study such as the consent form, send via email beforehand and ask them review and send back in lieu of a physical signature. If you don’t, be prepared to read aloud the consent form and assist in signing the documents for some participants.

Make sure directions provide step-wise directions; do not rely only on graphical maps as these may not be accessible.

For all documents, make sure color is not used as the sole cue to convey information. Print out all documents on a black and white printer to make sure color is not required to make sense of the information.

Get participants mobile phone numbers in advance and meet them at their drop-off point. Be prepared to guide them to the testing location. Review best practice for guiding blind individuals:

While Braille documents can be helpful for participants that read Braille, the time and cost involved may not be feasible. Furthermore, all blind people do not read Braille, especially people that have lost sight later in life. It is best to make sure all documents can be read via a screen reader. Unless you are sure if there are no accessibility issues avoid PDF documents and send out simple Word documents or text-based emails.

If participants bring guide dogs do not treat them as pets, they are working. Provide space for the dog and do not pet it unless the participant gives you permission.

Make sure to explain beforehand any sounds or noise that are or may be present in the room such as unique audio from recording software. This may avoid the participant from becoming startled or confused during the session.

Initially when I started to work with blind participants I was worried my choice in language might offend. However, over the years I have learned that most blind participants are fairly relaxed when it comes to speech. Therefore, during moderation do not be afraid to use phrases such as “see” or “look” and similar words when talking to blind participants; for example, “please take a look at the bottom of the page” or “what do you see in the navigation menu?” In my experience, blind participants will not be offended and will understand the figurative meaning rather than the literal meaning.

Test out all recording equipment/processes beforehand. Ensure all audio including both human speech in the room and audio/speech from AT such as screen readers will be recorded correctly. During testing of the equipment adjust the locations of the microphones for optimal recording.

Lessons Learned For Participants With Motor Disabilities

Motor disabilities refer to disabilities that affect the use of arms or legs and mobility. These individuals may need to use a wheelchair. Some people may not have full use of their hands or arms and cannot use a standard mouse and keyboard. These people may need to voice recognition software which allows to use voice input or use a special pointing device, for example, one that is controlled by their mouth.

In the directions, make sure the route is accessible and routes them via elevators rather than stairs. Also, if participants are driving note the location of accessible parking.

Note if doors have accessible door controls. If not you may need to meet the participant and guide them to the testing location.

Make a note of the nearest accessible restrooms to the testing location.

As with all participants with disabilities, it is best if they can bring in their own laptop with their assistive technology software installed and any other required assistive technology. However, in the case of participants (such as Adriana in Figure 3) that use voice recognition software such as Dragon Naturally Speaking this is critical because they have trained the software to recognize their voice.

Make sure the desk or table where the participant will be working can accommodate a wheelchair and the height is adjustable. According to the American with Disabilities Act (ADA), conference tables must be 27 inches high in order to accommodate knee clearance for individuals in wheelchairs..

Adriana Mallozzi conducting a usability test at the User Experience Center. Adriana has a motor disability (cerebral palsy) which affects her entire body. She uses a wheelchair, a sip-and-puff joystick that acts as a mouse, along with Dragon Naturally Speaking. (Large preview)

Lessons Learned For Participants With Cognitive Disabilities

Individuals with these disabilities cover a wide range of relatively mild learning disabilities such as Dyslexia to individuals with a more profound cognitive disability such as Down syndrome. In general, people with cognitive disabilities have challenges with one or more mental tasks. Rather than looking at specific clinical definitions it best to consider functional limitations in key areas such as memory, problem-solving, attention, reading or verbal compensation. Consider how best to accommodate participants during usability testing. Many of the tips below should also apply to all participants, however for this group you need to be extra aware.

Sometimes participants will be accompanied by a caretaker or an aide. This person may assist with transportation or may need to be present with the participant during the usability test. If the caretaker is present during the usability test, make sure they understand the structure of the usability test and what will be required of the participant. If you know the participant will be accompanied before the study, you review the goals and protocol prior to arrival via email or phone. That is as much as possible the participant should be one conducting the usability testing, and the caretaker should not be involved unless it is completely necessary.

In some cases, the caretaker or aide may act like an interpreter. You may need to communicate with this interpreter in order to communicate with the participant. If this is the case, make sure you record the audio coming from both the participant and the interpreter.

Provide instructions in multiple modalities, for example, both written and verbal. Be patient and be prepared to repeat the task or ask the same question multiple times.

Be prepared to break tasks into smaller sub-tasks to support memory/attention challenges or fatigue that may set in.

Ideally, it is best to be consistent with tasks for all participants however for some participants with cognitive disabilities you should be prepared to go off-script or modify tasks on the fly if the current approach is not working.

Have the participant’s comfort and well-being the number one priority at all times. Don’t be afraid to take multiple breaks or end the session early if things are just not working out or the participant is not comfortable.

Additional Insights

The tips above should serve as guidelines. Each participant is unique and may require various accommodations depending on their situation. Furthermore, while some of the tips are categorized for specific disability types, specific individuals may have multiple disabilities and/or benefit from a tip from a different category than their primary disability.

If you or your company have conducted user or customer research, you know the value of gathering feedback about the issues and benefits of products and systems. Testing with individuals with disabilities is no different, as you learn many insights that you would not gain otherwise. However, an additional takeaway for us was the realization that people use assistive technologies in different ways. The following example is specific to people with visual disabilities, but there are similar examples across all groups.

An assumption might be someone that is blind only uses a screen reader such as JAWS and is an expert at it. We found that people with visual impairments actually differ greatly in the level of support needed from assistive technology.

Some users need a screen reader for accessing all content.

Some users (with more sight/with low vision) only need to enlarge content or invert page colors to increase contrast.

Others may need a combination of approaches. One visually impaired participant used both a screen reader along with the zoom function embedded in the web browser. She only used a screen reader for large paragraphs of text, but otherwise simply zoomed in with the web browser and got very close to the screen when navigating around the website.

Furthermore, just like anyone, all users are not experts on the software they use. While some users would consider themselves experts, some only learn enough about the software to accomplish what they need and no more.

Moving Forward

Hopefully you have learned some useful information that will help you include more diversity into your usability testing. However, since there is variability with different disabilities, this may seem overwhelming. I recommend starting small; for example by including one or two participants with disabilities as part of a larger group of 5 to 10 participants. In addition, initially bring in someone that has both experience with usability testing and a lot of experience with their assistive technology so you can focus on getting their feedback rather than how the usability testing process works or their use of their assistive technology.

Acknowledgements

I would like to thank Jocelyn Bellas, UX Researcher at Bank of America and Rachel Graham, UX Researcher at Amazon. When Rachel and Jocelyn worked at the User Experience Center as Research Associates in 2016, they worked with me on some of the projects referenced in this article and also contributed to a related blog post on this topic.