+ Nima, Isabel, and everyone else who has been generously giving us feedback along the way. Thank you!

Possible User Paths

TODO.

Qualitative User Experience Research

Takeaways:

People come to their own (usually bad) conclusions about what to do next when the appropriate information is not available in the interface": When a connection fails, the user is redirected to the proxy screen, which is just the last screen of the launcher configuration dialog. Most users interpreted this as that the proxy was the source of the problem, and that they needed to configure a proxy somehow (when in reality, this was literally never the case for our simulations).

Users don't read the text much: People didn't read at all, or read the first sentence. If they got really desperate and failed a couple times, then they might read more.

Users get easily discouraged: People felt "sad" that they couldn't "make it work," and tried less and less with each failure. Getting it right the first time is pretty critical for user experience and user morale.

Users don't have the background to make these decisions: Most of our users did not have technical backgrounds. They didn't know the difference between a censored website or censored Tor relays, or what a bridge was. Most of them probably don't know how internet networking works, to be honest.

Sources of Failure (Tor Launcher v5.0.3):

Users lack mental model of censorship: On the first page of the launcher, it says "This computer's internet connection is censored .." Most of our participants just thought, well, I'm censored, because I can't go to this website, so I must configure this! This was the wrong decision for our participants in a censorship environment which only blocked websites, and led them to a complicated configuration process. This was the right decision for the participants in environments where Tor bridges were censored, but they did not make this decision knowingly. Most did not know the difference between a censored website or a censored network. The really technical participants were able to ask this question, but had no way to tell which type of censorship was happening.

Interface guidance doesn’t match the ideal path: The real estate spent on each screen didn't indicate the ideal path. For instance, more space on the bridge configuration screen was used for custom bridge configuration, even though that is the choice we want to de-emphasize. As mentioned above, the interface also guides users to the proxy screen after a connection error, even if the source of the error was not a proxy misconfiguration.

There’s no design for failure cases: Error messages were not helpful to the users, since most were too technical to be understandable. Additionally, it does not take the opportunity to suggest the user to do something else (wait more, choose another bridge, etc.)

P3: this participant configured a bridge successfully (obfs3, the default), even though they didn't need to. They didn't know the difference between censored websites and censored internet connections.

P11: this participant tries to connect using vanilla tor by clicking connect, which doesn't work against a censor that blocks public Tor relays. Then they try to connect using vanilla tor because they answered that they didn't need or a proxy, which also didn't work. They try connecting with a bridge (obfs3, default) and no proxy, which works.

P14: watch this poor user try the same configuration 7 times(?!!) and fail and guess proxy settings. 27 minutes, later, we cut them off. Because they are convinced they need a proxy, they failed to connect because they never bother to pick a different bridge (chose obfs3, required meek or custom).

Linda's favorite 3:

P1: this participant makes a correct configuration (obfs3, no proxy), but since bootstrapping took too long, was convinced it was wrong. Then tried "connecting directly" by pressing connect on the first screen, which worked. But the participant was convinced that they made a direct connection.

P4: after successful configuration, the participant is convinced that the file explorer on windows IS the Tor browser for a long time, before figuring out what the browser is.

P5: watch this participant create an email account on the fly and use the auto responder to use a custom bridge to connect successfully.

Methodology

To identify problems with the existing interface, we con- ducted a series of qualitative evaluations with a set of users representative of the general population. Using established best practices from the field of user experience research, we recruited five users for each censorship environment. We pre-screened our participants to have a good mix of gender, age, technical background, and familiarity with Tor in each environment, and overall.

We chose to simulate our censorship environments for the experiment. Although inspired by reality, the purpose of these environments are not to simulate a particular country, but to require distinct configurations of the Tor Browser from our participants (i.e. choose a meek bridge or is not able to connect). We find that simulation is far favorable over VPNing due to reproducibility of environment, risks to our users, and feasibility of technical implementation.

We simulated 3 censorship environments:

Level 1: websites are blocked

Level 2: websites are blocked, public tor relays are blocked

Level 3: websites are blocked, public tor relays are blocked, hard-coded relays in the interface are blocked.

Experimental Setup

The experiment begins with the participants being informed that they are in a simulated censorship environment, where some — but not all — websites are blocked. We instruct them to visit a non-blocked website and a blocked website on a “standard” browser (one that is not used for censorship circumvention, such as Chrome, Internet Explorer, or Firefox) to illustrate the situation. Participants are told that their role is that of a regular citizen who is trying to visit blocked websites for non-critical purposes, and for whom the chance of risk is minimal. This information is given to achieve consistency across participants: behaviors may change if the tasks are critical or if there are great risks associated with configuration mistakes.

After explaining the situation and the user’s role, we in- struct them to complete a set of tasks which will require visiting blocked websites. Ultimately, the participants will need to configure their Tor Browser to circumvent the simulated censorship environment. These tasks have a three-fold purpose of providing participants with feedback (if they did not correctly configure their browser, they cannot reach the website), motivating the participants to configure the browser correctly, and providing the researchers with an indicator of successful censorship circumvention. Participants’ screens are recorded to capture how they configure their browser.

After users completed the browsing tasks (or time ran out), we conducted an interview asking participants about their censorship circumvention experience. We were able to tailor the interview to the particular participant, since we were able to observe our participants’ sessions. This way, we could follow up and ask participants about reasons for taking particular actions and verify any hypothesis we had about the participant (such as “they didn’t know what to do on window 3”). We did ask a couple question across all participants: “What was the most challenging part of connecting?” and “What is one change you would recommend?”

Instrumentation

We borrowed a Windows computer from Department of Electrical Engineering and Computer Science at University of California, Berkeley. We surface-level sanitized the computer to not distract users with icons on the desktop, bloatware, etc. and installed necessary programs such as Tor. With user permission, we captured users' screens as they configured their browsers in various simulated censorship environments. For the duration of the experiment, we used the same computer for all of our participants. Note that all of our participants natively used windows, so there may be some friction for some users, but we don't believe that this had a significant impact on the results of our study.

We did not record any of the interview process as voice is personally identified information, electing to take notes on the follow-up interview instead. We did not keep any of the worksheets or record the worksheet information, as the purpose of the worksheet was to get user to visit websites which were blocked. We are able to verify if the user had successfully circumvented censorship through the screen recordings.

Results

We identified common mistakes that users made during the configuration process, and how common those mistakes were. Some findings:

Some were not able to connect successfully, and many configured a bridge when they didn't need to.

People don't know if they should click connect or configure. People don't know if the website is blocked or Tor is blocked.

People don't know how to choose between bridges, know what any of the bridges do. All went with the default bridge.

Many people thought that they needed a proxy when they didn't.

The connection process takes so long that people think that it failed, even if they picked the right configurations.

Since the progress bar only updates with each log message, if a participant tries once, gets to 50%, and connects with a different configuration, they will see a progress bar at 0% until their next connection gets past 50%.

Each subsequent attempt has a lower rate of success.

People didn't read much of the text in the interface.

People didn't have interest in learning what a bridge/proxy was, but rather what bridge/proxy would work for a connection.

Error messages were unhelpful.

More feedback and indicators that a connection is in progress would be helpful.

There are lots of ways that people can mess up.

Interface Designs

Clickable Prototypes

You can see the design decisions that we made from the results of our qualitative user experience research. We based each change on concrete events, trends observed, interview dialogue, etc.

Note: these clickable prototypes are rough, so there may be some missing features (i.e. if you click "help," nothing happens and there is no help window, but that's because we don't plan on changing that, nor did we take the time to re-draft it up). Additionally, there is only one version of each window in the prototype, so the state may seem inconsistent (i.e. in Iteration 4, no matter what bridge you choose, you end up choosing the same one). These were mainly to demonstrate the design changes that we want to make to the interface, so please bear with us! It is quite cool that you can click through the prototypes and explore the paths yourselves.

Tip: Marvel, the site where our clickable prototypes are hosted, will act as a prototype. You can explore the paths manually to iterate through all the windows, but if you want to see all the windows at once and navigate to any window, you can click this icon at the top right corner. It will then display a grid of each window of the prototype. Unfortunately, I can't find a way to display all windows at once bigger than the grid appears. But clicking on a window's thumbnail will navigate you directly to that window, without you having to traverse through the interface path yourself.

This is our first iteration of the prototype: just a lo-fi sketch to get out the ideas.

Changes from Iteration 1 -> Iteration 2

Finalize changes and decide on the style of the mockup. Transitioned from pencil to image-based mockups.

Changes from Iteration 2 -> Iteration 3

Made changes to the progress bar, and worked on simplifying choices for the user.

Changes from Iteration 3 -> Iteration 4

Updated text, agreed on final version of the progress bar and animating indicator.

Changes from Iteration 4 -> Iteration 5

We ran a focus group at Berkeley during one of the BLUES (Berkeley Laboratory for Usable and Experimental Security) Lab meetings. 14 researchers, ranging from a spectrum of computer science to pure user experience researchers were in attendance and provided feedback on the design. The discussion started with an overview of various censorship environments around the world, followed by a discussion of how the 3 simulated censorship environments in the lab were representative of these conditions, a walkthrough of Tor's current (5.0.3) version of the launcher and the ideal path for a user in each of the 3 simulated censorship environments would take, and the results of the qualitative user experience research. Then, we presented our new design (Iteration 4), stated our justifications for changes made from the original launcher to our new design, and concluded with a discussion of the design decisions made and possible improvements.

Auto-proxy detection: We plan to automatically and locally detect if a proxy is already configured on the machine. This can be done by looking for some configuration file for other browsers that users have installed. If we detect that a proxy is not necessary, we will inform the users that they do not need to configure a proxy, and show a window which encourages them to go on without configuring one (but they can still do so if they want to). This should help the many users who did not know if they need to configure a proxy or not.

Better User Workflow: Let's help our users make decisions with less friction. For instance, if a user tries to connect directly and fails, the options should be to retry or to go to the configuration screen, not the home screen. Similarly, if their connection failed due to a misconfigured proxy, they should be redirected to the proxy page. If their connection failed due to a misconfigured bridge, they should be taken to the bridge screen.

Proxy configuration before Bridge configuration: This was proposed for two main reasons. 1) the network topology connects to a proxy first and a bridge thereafter. so having the steps in that order may help build a user's mental model. Our progress bar at the connection screen shows progress in a network topological order, so this would also make our design more consistent. 2) for smarter redirections. If a connection fails because of a proxy error, the person may need to also configure a bridge. But if a connection fails because of a bridge error, then the users do not need to see the proxy screen again. In the current design where bridge configuration is first and proxy is second, a user would need to click back to configure a bridge if proxy error, and see an unnecessary proxy screen if they had a bridge error.

Eliminating enumeration: "Step 1", "Step 2", "Step 3" text at the progress bar in the configuration windows were deemed to be clutter-y, redundant, and a bit confusing. Configuring a bridge doesn't necessarily have to be done before configuring a proxy, and enumeration seems to imply that there need so to be a specific order to things. The little blue arrow shapes will be enough to indicate progress.

Visual guidance: We will highlight the paths that users should take in green (i.e. the "next" button will be the same color bright green as the "connect" button on the first screen). This helps user fatigue, as they don't need to read the words on the screen as carefully to determine which is the most obvious step to do next. We did this to minimize fatigue, since users already need to make a series of decisions and possibly need to troubleshoot their connections.

Changes from Iteration 5 -> Iteration 6 (Final)

Changes were made based on the feedback from Valencia 2016.

Fixing Bugs: Lots of little odd behaviors happened on the progress and summary screen with custom bridges, proxies without username and password fields, etc. Also fixed the code to ensure that navigation works like we intended (there was some difference between our mockup and how the implemented version behaved; now they are aligned).

Updating Language: using language to indicate the default settings (changing "recommended" to "default" to guide users better; specifically a suggestion from many UX researchers after discussion), proxy screen language says that proxy settings are detected locally, bridge screen tells users to select a bridge (users might not read the banner, so giving them instructions in text is required; this is from testing the interface on multiple people).

Streamlining Visuals: The custom bridges box pops up on selecting the custom bridge radio and is hidden otherwise, the proxy fields pop up on selecting to configure a proxy and is hidden otherwise, the progress bar updates itself with log messages and gives user feedback on where their connection is, there is an animation to indicate to the user that there is progress being made during the long connection, and buttons/text/banners are finalized to be ready to test.

Error Feedback: We planned to implement this all along, but we never got to doing it. We added messages for the user giving them suggestions on what to do if they encounter a particular error. Our animated progress bar also shows where the error occurred, visually (people know that they couldn't connect to the proxy, bridge, or Tor) and we translate the error messages for them in a way that they find helpful and they can understand ("try another bridge" over "error in making a connection to bridge").

No Bridge Option: The old interface didn't give a user a no bridge option if they chose to configure, since a bridge is what they should configure if they chose to configure their settings. But we realize that giving people the option to choose the default settings is wise, given that a lot of people did click configure when they should have clicked connect. We were torn on this, especially because a multiple participants' first attempt was to connect directly, then to configure, and connect with no bridge or proxy configured. We ultimately chose to give them the option to proceed without a bridge because it is not intuitive for the user to go back screens to choose a particular option.

User Acceptance Testing

Coming March 2016.

Methodology

We plan to recruit participants from the "general population" (read, not specifically recruiting technically inclined people, but not avoiding them either) from the San Francisco Bay Area (which has a unique demographic) to test our new interface design versus the current (5.3.0) Tor configuration interface. We are aiming for ideally ~120 participants, so that we can have ~20 participants per condition. The 6 conditions arise form the 3 levels of simulated censorship environments x 2 types of interfaces to test. For an hour of their time, participants will be compensated $30 in the form of an amazon gift card. There are no anticipated direct or indirect risks for our participants.

Experimental Setup

TODO

Instrumentation

We plan to instrument the Tor launcher (v 5.0.3) to greedily log events. From data such as mouse clicks, paths taken, total time to completion, we plan to analyze which features were helpful or harmful. The specific type of analyses are yet to be finalized--feel free to give us suggestions on what to look at.

Some of the ideas so far: number attempts to success, number of total configurations attempted, total time to completion, number of repeated attempts to connect with the same configuration, ...

Results

TODO

Discussion

"Are there consequences of automating your configuration?"

It IS possible to automate this entire process for our users. If we were only concerned about how easy it was for users to use the launcher, minimizing the time it takes a user to make a successful connection to the Tor network, or even the rate of overall success of making a connection to the Tot network, it would make a lot of sense to automate the entire thing. The system can auto-detect for proxies, and the system can auto-probe to see which bridges would work for that user. The talk of introducing some level of automation into the launcher has been talked about this within the Tor community, has repeatedly come up in discussions among Berkeley researchers, and suggested by members of the focus group which helped the transition from Iteration 4 to Iteration 5.

However, there may be situations where this can be dangerous. For instance, a government could log any unsuccessful attempts to connect to the Tor network and prosecute someone based on this information. To our knowledge, this is pretty rare today, but 1) we must consider the agency of our users 2) this may be a more threatening concern in the future.

If we just knew if our users would be safe if we automatically configured for them, we would! But there's no good way to do this. One straight-forward way to get this information is to ask the users, "Are there consequences of automating your configuration?", but this is imperfect in many ways. Firstly, users may not be aware that there is any danger presented to them. Secondly, users may deliberately and knowingly put themselves in danger. Yes, it would be their choice, but it may have ethical concerns if we put it in our interface as the path of least resistance.

Is there any way to get this information? Is this a good idea? How much of the Tor launcher should be automated (just the proxies, bridges, proxies and bridges)?

"Do you know which bridge to choose?"

An alternate question to ask our users is to ask them if they have enough information to make an informed decision. In the previous section, we asked them if they were in any danger. To ask them if they have enough information to make a decision is a different question which considers the ability of the user to make decisions, but does not take into account their consequences as directly. One could argue that people who don't know how to configure their connection would have made mistakes anyway, but making the mistakes for them is quite a different thing than letting them make the mistakes themselves.

In this version of automation is to auto-detect for proxies and only involve the user if a proxy is configured, and for the bridge configuration to ask the user if 1) they want to not use a bridge 2) If they want the launcher to automatically pick a bridge for them (get some permission to auto-probe for bridges). This would work, but will users have an accurate assessment of their technical knowledge? Would they answer truthfully?

Are proxies configured often?

Generally across the internet, the number of users who are behind a proxy are very, very few. Does this change for users of Tor?

If proxies are very, very, very, rarely configured, then we could automate the detection for if a proxy is required and skip the proxy configuration screen entirely, and only show it when it is necessary. Currently, all users see this screen at some point during the configuration, and during the qualitative user experience studies, proxies were the largest source of confusion.

A More Enticing "Help" Button

In our qualitative user experience studies, we found that not many people clicked on the help button. When asking some of our users about why they didn't, one responded that the help button is 1) usually not useful, 2) a huge block of text, and 3) requires more work from the user. The participant responded that if a help button would actually "help," or perform something on behalf of the user, then the help button would be more useful, and tempting to click on.

Currently, the help button is only on the bridge configuration window, and it helps users enter custom bridges. What about expanding the help button to include other sources of confusion, or making it more useful for the user? I (Linda) don't have any concrete ideas on specific things that the help button should do or should not say, but I think that a discussion could be fruitful.

Miscellaneous Considerations

Some of these points were brought up at one point or another. We didn't find them high-priority, but we didn't want to lose them either, so they live here.

Examples of configuration: Would giving examples of configurations in China, Iran, France, etc. help a user?

Should the help button be renamed to something like "I'm Stuck!" or something more enticing?