Skip to main content

Delicious Reverie

blog of developer & bookworm benjamin read

Automated accessibility testing is great, but ...

We recently had a situation using jest-axe in tests, where the landmarks feature wasn't work as expected. This helped me verify why we shouldn't rely solely on automated tools, never mind how good they are.

In unit tests for our react component library at Zopa, we have utilised jest-axe to test whether our components adhere to a number of accessibility rules.

The funny thing was, when we adhered to the rules (specifically the way components either use semantic HTML or aria roles), further testing in Google's Lighthouse tool revealed that there were accessibility issues with our components:

Some ARIA child roles must be contained by specific parent roles to properly perform their intended accessibility functions.

The trouble is that our library contains a lot of components that do not require the use of landmarks. We build our UI pages up from these component pieces, and would utilise landmarks on the page typically.

This rule was forcing our developers to add aria roles where they weren't needed.

Thankfully, You can disable this rule. However, this means that there are far less checks for semantic HTML in our codebase.

I don't think this kind of issue is something that's easily solved by the axe team. It does highlight the flaws in tools like jest-axe: we can't rely solely on them. In fact, right on the Github repo the team identify clearly (in large writing!) that " This project does not guarantee what you build is accessible", and that "The GDS Accessibility team found that only ~30% of issues are found by automated testing."

We ultimately decided that ~30% is better than nothing. And <div> soup, whilst undesirable, is better than using roles where they're misleading. So we disabled this rule and we're going to continue to use jest-axe in our project.

Other ways forward

To try to keep our focus on giving all of our users the best experience possible, we have recently created the role of "feature champions" among the frontend community.

These individuals take on a subject that they are passionate about, such as TypeScript, unit testing and, yes, accessibility too, and will occasionally give a quick eye to projects to kindly advise whether we are following best practices.

So much more is needed. Each of us needs to continue to develop empathy for our customers and visitors. And we need to avoid trusting the tools we use for accessibility: they don't always get everything right.


You're currently on the javascript disabled version of the site. To enable the site search and some pretty animations, view the javascript enabled react app.

About This Site

Delicious Reverie is the blog of developer & bookworm benjamin read. Set in Skybird and your system font. Built with Gatsby, hosted by Netlify. © Some rights are reserved. This site doesn't use any cookies or other session storage and has no tracking scripts.

Where you can find me: