TalkBack & Magnification Accessibility in Android 4.3+

Table of Contents

Abstract

Is Google closing the gap on Apple's mobile accessibility features? Now Android has magnification support for low vision users! With Android's open app ecosystem you can even install multiple accessible web browsers like Chrome & Firefox. See & Hear a demo of the latest assistive technologies in Android JellyBean 4.3+.

Slideshow URL

Presenter

Slideshow

Press the Left/Right Arrow Keys or Page Up/Down to navigate through slides, the Home key takes you home to the table of contents. Adjust text size in the Info dialog. Share this presentation on Twitter & Facebook.

This slideshow has been developed with jQuery Mobile to create a universally accessible presentation that will work on any accessibility enabled device. PowerPoint & PDF formats are not universally accessible.

TalkBack

TalkBack is an Accessibility Service that helps blind and vision-impaired users interact with their devices more easily.
This application add spoken, audible, and vibration feedback to your device. It is a system application that was pre-installed on most device and is updated when the accessibility service is improved.
This app is only activated if you explicitly turn on Accessibility.
Steps to activate Accessibility:
1. Go to Settings
2. Select Accessibility
3. (Android 3.2 and earlier) Enable Accessibility checkbox
4. Enable TalkBack checkboxes
5. (Android 4.0 and later) Enable explore-by-touch

Vibration Feedback (Replaced Kickback)

Accessibility Shortcuts

Explore by Touch

Explore by touch tutorial is basically the built in TalkBack user guide.

Magnification

The keyboard does not get magnified.

Enable screen magnification to easily zoom or pan the entire screen to get a closer look. Visually impaired users can now enter full-screen magnification with a triple-tap on the screen, and even type and interact with the device while zoomed in.

Some or all of this information applies only to Nexus 7 devices running Android 4.2.

When this feature is turned on, you can temporarily magnify what’s on your screen or use magnification mode to easily zoom and pan your screen. (For users with low vision)

Note: Triple-tap for magnification works everywhere except for the keyboard and navigation buttons.

Temporarily magnify: Triple-tap & hold.Magnify & pan: Triple-tap & hold, then drag your finger.Toggle magnification mode in or out: Triple tap & release, or enter or exit an app to get out of magnification mode.
While you're in magnification mode, you can:Pan: Drag two or more fingers across the screen.Adjust zoom level: Pinch or expand using two or more fingers.

General Android Accessibility Recommendations

Fat Fingers - Use recommended touch target sizes

Buttons larger than the minimum recommendations are appropriate for children with developing motor skills and people with manual dexterity challenges.

Label visual UI elements meaningfully

Label functional UI components that have no visible text: buttons, icons, tabs with icons, and icons with state (like stars).

Use the contentDescription attribute to set the label.

group

all contacts

favorites

search

action overflow button

when starred: remove from favorites
when not starred: add to favorties

action overflow button

text message

Provide accessible alternatives for timed-out controls that disappear

Icons or controls that disappear after a certain amount of time, e.g. Video player controls that fade out after 5 seconds.

If controls fade out before a user focuses on them then a TalkBack user may not know they were there.

You can also change the behavior of your app when accessibility services are turned on, e.g make sure that timed-out controls won't disappear.

Use standard framework controls or enable TalkBack for custom controls

Standard Android framework controls work automatically with accessibility services and have ContentDescriptions built in by default.

User controlled system-wide large font size in Settings; using the default system font size in your application will enable the user's preferences in your app as well. Mark text and their associated containers to be measured in scale pixels.

Your app may not have room for a user's large fonts or language settings, the text may get too large for the containers that hold it and be cut off.

Try TalkBack Yourself!

Turn on the TalkBack service in Settings > Accessibility and navigate your application.

Browser & Web View App Accessibility

Android has good support for WAI-ARIA and HTML5 Accessibility.

I think it may be possible to set a web view app to render with Mozilla's Geko (Firefox) browser engine rather than Chrome's. Firefox web views are more accessible than Chrome.

Captions

Timed text tracks

The MediaPlayer now handles both in-band and out-of-band text tracks. In-band text tracks come as a text track within an MP4 or 3GPP media source. Out-of-band text tracks can be added as an external text source via addTimedTextSource() method.

Android 4.4 KitKat Closed Captions API

Android 4.4 now supports a better accessibility experience across apps by adding system-wide preferences for Closed Captioning. Users can go to Settings > Accessibility > Captions to set global captioning preferences, such as whether to show captions and what language, text size, and text style to use.

Testing for Accessibility with TalkBack

All task workflows in the application can be easily navigated using directional controls and provide clear and appropriate feedback.

Directional controls: Verify that the application can be operated without the use of a touch screen.
Note: Keyboards and D-pads provide different navigation paths than accessibility gestures. While gestures allow users to focus on nearly any on-screen content, keyboard and D-pad navigation only allow focus on input fields and buttons.

TalkBack audio prompts: UI controls have clear and accurate audio descriptions when TalkBack is enabled and controls are focused. Use directional controls to move focus between application layout elements.

Explore by Touch prompts: UI controls have appropriate audio descriptions when Explore by Touch is enabled. There should be no regions where contents or controls do not provide an audio description.

Touchable control sizes: All controls where a user can select or take an action must be a minimum of 48 dp (approximately 9mm) in length and width, as recommended by Android Design.

Gestures work with TalkBack enabled: Verify that app-specific gestures, such as zooming images, scrolling lists, swiping between pages or navigating carousel controls continue to work when TalkBack is enabled. If these gestures do not function, then an alternative interface for these actions must be provided.

No audio-only feedback: Audio feedback must always have a secondary feedback mechanism to support users who are deaf or hard of hearing, for example: A sound alert for the arrival of a message should also be accompanied by a system Notification, haptic feedback (if available) or another visual alert.

Repetitive audio prompting: Check that closely related controls (such as items with multiple components in a list) do not simply repeat the same audio prompt. For example, in a contacts list that contains a contact picture, written name and title, the prompts should not simply repeat “Bob Smith” for each item.

Audio prompt overloading or underloading: Check that closely related controls provide an appropriate level of audio information that enables users to understand and act on a screen element. Too little or too much prompting can make it difficult to understand and use a control.

Basic TalkBack Testing Methods

Enable TalkBack

Check the Name, Role, Value, State of each element.

Set focus to all UI elements in the app and make sure the accessible name (contentDescription) makes sense. Make sure text alternatives for images match fully.

If UI element is actionable should it have a role like button or tab? Static text would not have a role. (Called traits on iOS, role in WAI-ARIA)

Adjustable controls like sliders must have their current Value spoken to TalkBack.

Is the current state of a control spoken? E.g. is this tab the selected tab? Is this element expanded or collapsed?

Images that are purely for decoration (have no function) should not be focusable.

Is there proper focus management when new content appears on the screen? E.g. when a dialog appears does the TalkBack focus go into the dialog? Is it trapped in the dialog? Or is the focus still stuck underneath the dialog? Is focus lost when the dialog is closed? Does focus return to the UI element that opened the dialog? When error messages appear are the spoken automatically by TalkBack?

Is the focus order logical when swiping through elements or using directional controllers? Generallly follow the visual reading order, left to right, top to bottom.

Is the TalkBack user aware of gestures specific to your app that they need to know to activate content or features? If those custom gestures conflict with TalkBack is there an alternative method to acheive the same function?

The emoticon image should not be focusable because its text alternative is already adjacent (Feeling great). Unless you want to get more descriptive of the icon's facial expression. This image has no contentDescription so TalkBack speaks nothing when it's focused on. Focus should be removed.

When there is a number of notifications, TalkBack reads 3 different focusable elements including the number “twenty” as a separate element.

10 plus new stories has no button role

Under the main navigation menu each settings cog image button is unlabeled but should not likely be focusable separately from the table cell button, if it remains focusable then it should have a proper contentDescription

When the main navigation menu and contacts menu are expanded the Facebook stream content is still focusable with TalkBack and can be scrolled down. This is not the case for sighted users where the menu is treated as modal content.

Status, Photo, and Check In buttons are skipped over during linear swipe navigation. They can be focused on with explore by touch but they have no button roles.

Each person’s image and name are both focusable in the stream. The image element has no image role. It may be smarter to remove the focusability of the image so TalkBack does not announce each person’s name twice.

There is a “menu button” for each post in the stream. I would help to have a more descriptive accessible name for those “menu button”

Every time TalkBack user gets to the end of the stream and swipes to the next element loading in the next view of the scroll area, after that view loads it reads “showing items x of x” but then when I swipe again my focus has been lost and now I’m focused at the very top on the main navigation button.

The text “10 likes 3 comments” has no button role so how does a TalkBack user know if it’s static text or actionable button elements?

When I activated the 10 likes 3 comments modal dialog, TalkBack focus did not go into the dialog. Focus is still underneath the dialog.

Elements like “load previous comments” that are buttons but have no button roles make it hard for TalkBack user to know whether an item is static text or actionable buttons.

In the comments dialog there is a blank “space” element spoken to TB when each com mentor’s name is focused on.

In comments dialog the like button has no button role

In comments dialog focus goes to the commenter’s thumbnail image but nothing is spoken to TB, might be best to remove TB focus.

Post button under write a comment has no button role

Under write a comment I’m able to get TB focus to the main view hidden underneath it when this should not be focusable

No text alternative for the loading progress indicator when a dialog was displayed.

In linear swipe navigation the TB focus skips right over the images inside friend’s status updates

There’s no image roles spoken to TB when focused on an image

There’s no indication when an image is full screen (expanded) or thumbnail size (collapsed) I’m not saying you have to add that text but there should be some text spoken to TB so the user knows when they’re in the 2 different image/thumbnail states

The full screen image is not really focusable with TB, no accessible name is spoken

When full screen image is displayed the TB focus still goes to content underneath the image.

You could possibly send an announcement to TB when the user hits the like button and the 1 like text animates onto screen so the TB user hears “1 like”, consider if you think it’s useful or if you think they should manually go check the number of likes. Something to think about.

In the People who like this dialog the 53 mutual friends text is read before the person’s name. The friend checked and friend add buttons are both unlabeled. The accessible name for each cell is just 53 mutual friends rather than the person’s name. There’s a < back arrow button in the people who like this dialog but it’s not focusable to TB. There are many ways for a sighted user to escape a dialog like this one, they can tap outside of it, they can drag it down, they can hit the < back arrow button, or hit the System Back button. A TB user can only hit the System Back button because all of those other methods are not focusable and have no accessible names/roles. I’m not saying you have to make all methods available to TB users, but at least one other than the system back button.

In the likes and comments dialog the top left Like button has no state (whether it’s been liked or not liked)

Under the What are you doing? screen the delete button for something like a Feeling great emoticon has no label.

The Feeling great emoticon is focusable but does not need to be since it’s text alternative is already present, this image is considered decorative so it should not be focusable.

Native App Accessibility

Labeling UI Elements

Use the android:contentDescription attribute to label the ImageButton, ImageView, CheckBox and other user interface controls.

Provide an android:hint attribute instead of a content description for EditText fields.

Associate an android:hint attribute with any graphical icons used by controls that provide feedback to the user (for example, status or state information).

EditText

Note: For EditText fields, provide an android:hint attribute instead of a content description, to help users understand what content is expected when the text field is empty. When the field is filled, TalkBack reads the entered content to the user, instead of the hint text.

contentDescription on an EditText control is only read before text is typed into the field, once the field is emptied the contentDescription is no longer read. This is a reason why a hint is a better option. Even though both disappear as the accessible name once text is typed into the field the hint comes back into view when the field is emptied and it will be read again by TalkBack, not the case for contentDescription on EditText controls.

Dynamic Labels Use runtime method .setContentDescription()

Base the content description on some context, such as the state of a toggle button, or a piece of selectable data like a list item. To edit the content description at runtime, use the setContentDescription() method:

Don't overdo your labels

Avoid the web-developer pitfall of labelling everything with useless information. For instance, don't set an application icon's content description to "app icon". That just increases the noise a user needs to navigate in order to pull useful information from your interface.

Design for Focus Navigation (Directional D-Pad, & Keyboard)

Enabling view focus

setFocusable()
isFocusable()
requestFocus()

If a view is not focusable by default, you can make it focusable in your layout file by setting the android:focusable attribute to true or by calling the its setFocusable() method.

Controlling focus order

Each UI control has 4 attributes, android:nextFocusUp, android:nextFocusDown, android:nextFocusLeft, and android:nextFocusRight, which you can use to designate the next view to receive focus when the user navigates in that direction.

Note: You can modify the focus order of user interface components at runtime, using methods such as setNextFocusDownId() and setNextFocusRightId().

Building Accessible Custom Views

If your application requires a custom view component, you must do some additional work to ensure that your custom view is accessible. These are the main tasks for ensuring the accessibility of your view:

Handle directional controller clicks

Implement accessibility API methods

Send AccessibilityEvent objects specific to your custom view

Populate AccessibilityEvent and AccessibilityNodeInfo for your view

Fire Accessibility Events

If you write a custom view, make sure it fires events at the appropriate times. Generate events by calling sendAccessibilityEvent(int), with a parameter representing the type of event that occurred.

As an example, if you want to extend an image view such that you can write captions by typing on the keyboard when it has focus, it makes sense to fire an TYPE_VIEW_TEXT_CHANGED event, even though that's not normally built into image views. The code to generate that event would look like this:

// Do something nifty with this text, like speak the composed string
// back to the user.
speakToUser(eventText);
...
}

Providing a customized accessibility context

There are some cases where accessibility services cannot get adequate information from the view hierarchy. An example of this is a custom interface control that has two or more separately clickable areas, such as a calendar control. In this case, the services cannot get adequate information because the clickable subsections are not part of the view hierarchy.

In order to provide a virtual view hierarchy for a view, override the getAccessibilityNodeProvider() method in your custom view or view group and return an implementation of AccessibilityNodeProvider.

Special Cases and Considerations

Text field hints: For EditText fields, provide an android:hint attribute instead of a content description, to help users understand what content is expected when the text field is empty and allow the contents of the field to be spoken when it is filled.

Custom controls with high visual context: If your application contains a custom control with a high degree of visual context (such as a calendar control), default accessibility services processing may not provide adequate descriptions for users, and you should consider providing a virtual view hierarchy for your control using AccessibilityNodeProvider.

Custom controls and click handling: If a custom control in your application performs specific handling of user touch interaction, such as listening with onTouchEvent(MotionEvent) for MotionEvent.ACTION_DOWN and MotionEvent.ACTION_UP and treating it as a click event, you must trigger an AccessibilityEvent equivalent to a click and provide a way for accessibility services to perform this action for users. For more information, see Handling custom touch events.

Controls that change function: If you have buttons or other controls that change function during the normal activity of a user in your application (for example, a button that changes from Play to Pause), make sure you also change the android:contentDescription of the button appropriately.

Prompts for related controls: Make sure sets of controls which provide a single function, such as the DatePicker, provide useful audio feedback when an user interacts with the individual controls.

Video playback and captioning: If your application provides video playback, it must support captioning and subtitles to assist users who are deaf or hard of hearing. Your video playback controls must also clearly indicate if captioning is available for a video and provide a clear way of enabling captions.

Supplemental accessibility audio feedback: Use only the Android accessibility framework to provide accessibility audio feedback for your app. Accessibility services such as TalkBack should be the only way your application provides accessibility audio prompts to users. Provide the prompting information with a android:contentDescription XML layout attribute or dynamically add it using accessibility framework APIs. For example, if your application takes action that you want to announce to a user, such as automatically turning the page of a book, use the announceForAccessibility(CharSequence) method to have accessibility services speak this information to the user.

Custom controls with complex visual interactions: For custom controls that provide complex or non-standard visual interactions, provide a virtual view hierarchy for your control using AccessibilityNodeProvider that allows accessibility services to provide a simplified interaction model for the user. If this approach is not feasible, consider providing an alternate view that is accessible.

Sets of small controls: If you have controls that are smaller than the minimum recommended touch size in your application screens, consider grouping these controls together using a ViewGroup and providing a android:contentDescription for the group.

Decorative images and graphics: Elements in application screens that are purely decorative and do not provide any content or enable a user action should not have accessibility content descriptions.

Building Accessibility Services

An accessibility service is an application that provides user interface enhancements to assist users with disabilities, or who may temporarily be unable to fully interact with a device. For example, users who are driving, taking care of a young child or attending a very loud party might need additional or alternative interface feedback.

Android provides standard accessibility services, including TalkBack, and developers can create and distribute their own services. This document explains the basics of building an accessibility service.

There are no traits, no hints on all elements, etc. If you want a UI element to be spoken as a "button" to TalkBack then you must use a button or image button element since there is no button trait(role).

No accessibilityViewIsModal like iOS, the API is just not nearly as robust!

EditText elements can't have both a contentDescription and a hint! And as soon as you type some text the hint or contentDescription is NOT spoken.

Android handles links embedded in a textview by using an earcon ding but no role/trait to indicate the separate elements. They're not separately focusable!

Opening local context menu gives ability to activate separate links but this is not very obvious or documented much.

TalkBack users do not hear if something is actionable unless it's a button or input element.

Tab controls do not indicate a tab role or a selected state.

Common Questions/Issues with Android Native Accessibility

How do I hide an element from TalkBack so that it does not receive focus and no accessible name is announced?

What's New in Android 4.4 KitKat Accessibility API

Android 4.4 now supports a better accessibility experience across apps by adding system-wide preferences for Closed Captioning. Users can go to Settings > Accessibility > Captions to set global captioning preferences, such as whether to show captions and what language, text size, and text style to use.

Enhanced Accessibility APIs

Android 4.4 extends the accessibility APIs to support more precise structural and semantic description and observation of onscreen elements. With the new APIs, developers can improve the quality of accessible feedback by providing accessibility services with more information about on-screen elements.

In accessibility nodes, developers can now determine whether a node is a popup, get its input type, and more. You can also use new APIs to work with nodes that contain grid-like information, such as lists and tables. For example, you can now specify new supported actions, collection information, live region modes, and more.

New accessibility events let developers more closely follow the changes that are taking place in window content, and they can now listen for changes in the touch exploration mode on the device.

Fragmentation

Buy your phone directly from Google if you want to get the latest system updates without having to wait for your cellular carrier to approve them.

If you buy an Android phone from HTC it has the HTC Sense custom UI which may have a broken accessibility experience, same problem with Samsung TouchWiz. If you have one of these carrier phones you'd have to install a custom ROM to get to a stock Google UI.

Info

This presentation and slideshow template is always evolving, most ALT text should be added and correct. Please contact me if there are any issues.

Press the Left/Right Arrow Keys or Page Up/Down to navigate through slides, the Home key takes you home to the table of contents. If you're using a screen reader you may need to use the pass through key first then hit arrow keys. Page up/down works with screen readers enabled without the need for a pass through key.