Need to perform accessibility testing on your mobile and web apps? Or maybe you want to check your website’s accessibility score? Perhaps you’re not even sure how accessibility fits into the quality assurance world and want some more information about it. Whatever it is, you’re in the right place.
One of the essential things in the accessibility world, or a11y world for short, is WCAG – Web Content Accessibility Guidelines. In our guide to building accessible digital products, you can find a detailed breakdown of WCAG levels and principles. You can also find out how the initiative started at Infinum and, hopefully, become inspired to introduce it in your workplace.
In short, WCAG is the gold standard for anyone who needs a list of recommendations to make web content more accessible. It can be a little intimidating when you see it for the first time. It can be downright scary if you are a software tester who needs to go through the guidelines and implement them. No worries, we’re here to help you break down WCAG and figure out where to start.
Is my app good or bad?
If your team still hasn’t started with a11y, and you think it’s high time to do so, you can always do a quick review with one of these tools and estimate your app’s state. There are many accessibility testing tools, and they are all quite similar but present the results differently.
Some of the tools that we have tried are:
Desktop apps
Wave, AccessScan, and Lighthouse (on Chrome) – These tools can give you a quick overview of whether the guidelines are followed correctly or incorrectly, and some of them (such as Lighthouse) give you a score for your website
Android apps
Accessibility Scanner – This app allows you to scan screen by screen to catch problems related to contrast, touch targets, or missing labels
iOS apps
Accessibility Inspector – This is similar to the Accessibility Scanner; the only difference is that because of the iOS privacy policy, you have to use a simulator in Xcode
All of these tools can find accessibility issues or rather, specific errors regarding WCAG compliance. However, they sometimes struggle with finding logical problems or flows. Because of that, accessibility testing tools do not guarantee 100% compliance, and you should still manually check the guidelines in your app.
How do I choose a WCAG level?
WCAG outlines guidelines for three levels of accessibility, A, AA, or AAA, across four categories, perceivability, operability, understandability, and robustness. Explore the breakdown of the levels to understand what each of the levels entails.
| A | AA | AAA | |
| Perceivable | Content must be presented in a way that can be perceived by the majority of users, including those with visual, auditory, or other types of disabilities. | Content must be presented in a way that can be perceived by a wider range of users, including those with color blindness or impaired vision. | Content must be presented in a way that can be perceived by the widest range of users, including those with more severe disabilities. For example, sign language interpretation should be provided for all audio content. |
| Operable | Content must be presented in a way that can be operated by the majority of users, including those with visual, auditory, or other types of disabilities. | Content must be presented in a way that can be operated by a wider range of users, including those with visual or cognitive disabilities. | Content must be presented in a way that can be operated by the widest range of users, including those with more severe disabilities. |
| Understandable | Content must be presented in a way that can be understood by the majority of users, including those with cognitive disabilities. For example, instructions should be clear and concise. | Content must be presented in a way that can be understood by a wider range of users, including those with limited literacy or who speak languages other than the primary language of delivery. For example, abbreviations and acronyms should be defined. | Content must be presented in a way that can be understood by the widest range of users, including those with more severe cognitive disabilities. |
| Robust | Content must be presented in a way that is compatible with the majority of users, including assistive technologies. For example, all HTML code should be well-formed and valid. | Content must be presented in a way that is compatible with a wider range of users, including assistive technologies. For example, all content should be accessible without relying on any specific technology. | Content must be presented in a way that is compatible with the widest range of users, including assistive technologies. For example, all content should be accessible using a wide variety of technologies and devices. |
Here are some things to keep in mind when deciding upon the WCAG level you will be conforming to:
What is required by law
Every country has different regulations for public and private apps and websites
What the client expects
Sometimes clients already know which level of a11y they want to follow
What the company guidelines prescribe
Maybe there are certain accessibility requirements already in place at your company
If nothing here applies to you, have a chat with your team, check out the descriptions of the WCAG levels, and then decide.
The tools and tests that you’ll encounter the most
Keyboard navigation
Navigate through a page using only the tab button, shift+tab, arrows, and escape.
Inspecting the code
When you inspect the code, check if images and links have alternative text. This can also be done using some of the scanning tools mentioned earlier in the article.
Screen reader
This technology helps people who have difficulties seeing to access and interact with digital content, like websites or applications, via audio or touch. Use it to find out how users perceive a web page when using this assistive technology.
Switch access/Switch control
This assistive technology device replaces the need to use a computer keyboard or a mouse. People with both physical and cognitive impairments use it.
Manual testing
Some guidelines will need manual testing (or can be checked with both manual testing and other tools). For example, check if all audio-only media have an alternative for users who don’t use audio, ensuring that important information is not communicated in only one way. You can also check whether the app works across different platforms and OS versions.
Exploratory testing
Just as with regular testing, a great way to explore how users navigate the app with different tools is to turn on that tool and start exploring the whole app yourself. Don’t just go through defined flows; try looking around, clicking on stuff, playing media, etc.
When testing and reviewing the guidelines, note down all your findings. Agree with your team about how you will report the bugs you find. Will every screen/page have its task? Will every bug go into a separate task? Will you have one document detailing everything? Find out what makes the most sense to your way of working and open the tasks accordingly.
Writing test cases
After you decide on a WCAG level, your next step is to go through the guidelines and figure out how to actually perform accessibility testing. There are no shortcuts here. You’ll have to invest time to understand the guidelines (and test them later).
An example from WCAG guidelines:
How to Meet No Keyboard Trap (Level A) If the keyboard focus can be moved to a page component using a keyboard interface, then the focus can be moved away from that component using only a keyboard interface. If it requires more than an unmodified arrow or tab keys or other standard exit methods, the user is advised of the method for moving focus away.
This one is self-explanatory. The way to test this is to go through the whole app using only a keyboard, ensuring you can navigate through the page using the tab, arrow, enter, and escape keys, checking that it behaves as it should. In our handbook, you can find a detailed explanation of how to do this for mobile apps.
After you understand how to test everything, write some test cases to help you.
You can go into different levels of detail:
General overview
Write a test case for keyboard navigation. You can go through the whole app and write down all errors related to keyboard navigation
Flows/screens
Write a test case for flows or screens and write down all the errors found per page (such as keyboard navigation, alternative text, or screen reader issues)
More detailed test cases
Write a test case for every block/screen and divide them using different testing methods (e.g., keyboard navigation – header block)
Define expectations
In some cases, it’s clear what the expected result is. But there are some cases when we’re unsure how something should behave. Imagine you have some animated text that changes every few seconds.
For example: “Our company can help you with design.” The word “design” changes to “development,” “planning,” “managing projects,” and so on. Is the screen reader supposed to stop and read everything? Only the first option? Skip the section altogether?
Sometimes we’re unsure of what is expected, which is completely normal. If that happens, talk to the project team, especially the developers. What are our options? What are we able to do in this case? Talk to clients. What would they like to happen? Is there a perfect flow to tackle this? Find out.
Define responsibilities
During the testing process, you may find yourself in a situation where you’re unsure about who’s responsible for solving a particular issue.
For example, we’re building a website and find that the alternative text is missing. A content manager or copywriter would normally be your go-to person for this, but what if you don’t have anyone in those roles? Whose responsibility is it? This is something you should agree on within your team.
Or what if you encounter a bug? Is it a bug in our app? Or could it be a browser bug? Since I was using a screen reader, it could also be a screen reader bug. Some investigation would be needed, and it would probably require more profound development skills. If you’re not that tech-savvy, always talk to developers. They can help you dig around and find the source of a bug.
If your app or website is done, and the client is responsible for content input, you should talk to them and provide some guidelines on how to go about that. For example, do you have a field for inputting alternative text with every image? Explain to them how it works and why it’s important. Sometimes, even when you feel like you’ve covered everything, it won’t be perfect at first. That’s completely normal and a part of the process. Do your best, but accept that sometimes things will be out of your hands, and that’s ok.
Testing for accessibility issues is a steep learning curve
Accessibility is a team effort. Everyone should be on the same page about making an app accessible for us to start testing it and reporting bugs. If we’re not aligned in our expectations, there is no point in reporting issues no one will fix.
The process comes with a steep learning curve, so don’t feel bad if it takes you some time to learn how to perform accessibility testing and how to operate the tools. It’s normal not to have all the answers, but we have to help each other if we want to embrace this initiative.
After all, we all have the same goal – to improve the product and the user experience for all.
