Make invitations to beta services better

If you are not careful you can antagonise users before they have even started testing your service.

A middle stage in our process of developing digital services is known as private beta. We put the service online but people can only use it if they enter a code we provide. This enables us to limit access to only a certain number, or types, of invited people. We can then get feedback from them and make improvements before making it available to everyone (public beta).

I got to test the Apply for faster entry to the USA service during private beta with a person who had an iPad and limited experience of using websites. Such people are hard to find, especially in the target audience for Global Entry, which is the name of this service.

A research participant entering an invitation code on an iPad

I emailed them a link to the service and a 12 character beta code, such as: DSVN4Z5YEMST

Observations

It took them three frustrating minutes to enter the beta code correctly. With the consent of the participant, here's what happened:

They opened my email and tapped the link to the test website

When prompted to enter the invitation code they had to switch back to the email app

Not knowing about (how to) copy and paste, they got up to get a notepad and pen

They wrote down the beta code, complaining about how long and complex it was – anticipating problems

They typed in the code. Every letter was upper case so they had to tap the Shift key 10 times and the letter 10 times

There were two digits in the code so they switched the keyboard mode between letters and numbers four

When they pressed the Submit button an error message appeared stating that a beta code was required – instead of a message to say that an invalid code had been entered

They switched back to the email app and realised that they had made two mistakes when copying out the code

Back on the website, they went through the rigmarole again

Mercifully, this time was successful

As well as being very annoying, experiences such as this have the potential to put people in a bad frame of mind and so affect their impressions and actions when using the actual service. This will contaminate our research findings.

Reflections

Reflecting upon this I thought how silly it was to put such a complicated, error-prone barrier in front of our carefully crafted service. It had taken valuable time and patience from my participant to negotiate it. The invitation code entry is a temporary irrelevance and so should be as unobtrusive as possible.

During other tests, I was able to prepare using my laptop and so could quickly paste in the code for the participant when necessary. But here, at an impromptu test, or remotely with the user's hardware, that was not possible/appropriate. We prefer to run tests on users' devices because it's much closer to reality.

Recommendations

I thought about how the process could be smoother in such situations. Here are some ideas:

Use a link instead

Include the code in the link to the service, provided when inviting users. The validity of this code could be limited to a length of time or number of uses. So permitted users would go straight in without entering a code. If a link is not possible, consider these options:

Make the code easier to enter

In this example, it took at least 24 screen taps to enter a 12 character code. We could reduce this by:

Using just enough characters to get the number of combinations required

Using only letters or numbers. If you must use both, put the numbers at the end to minimise the need to switch keyboard type. You could also set the keyboard type automatically

Providing the code in upper case for readability but letting the user know that lower case can be used when typing it

Avoiding the use of similar characters, such as 1, I and l; O and 0; B and 8. Also 6,5 and 9 can be hard to tell apart for those with a visual impairment

Using simple words or even a phrase, such as ‘big red car’, instead of a code. There are libraries for developers to generate such things

Use an identifier instead

Ask the user to enter their email address, or some other value(s) known to both you and them, instead of an invitation code.

Of course, you could combine some of these ideas, too.

Conclusion

Invitation codes are a volume control, not a security mechanism. Maximise the usefulness of the private beta phase for the project team and, ultimately, the public by minimising the user's interaction required with the invitation code.