A guide to add auto accessibility check in storybook

It's always good to have a proper accessibility check when you are developing front-end components.

When developing with storybook, you can even get more: a pipeline to do auto checks for accessibility!

First, install a11y-addon

Storybook has an official addon called @storybook/addon-a11y, you can view more details from the official guide.

After installed, you can see a tab inside the storybook, shows various accessibility errors or warnings.

Hurray!!

Do this even better

One shortcoming of this approach is, the addon itself doesn't throw any errors. When multiple developers are working on several components, it is easy to overlook those a11y warnings (or even ignore it on purpose).

How to check it automatically and throw an error when pushing to the repository?

A pipeline to automatically check accessibility

Here I am going to use my storybook demo in Github with CircleCI.

If you aren't familiar with the CircleCI pipeline, see official documentation.

First, you have to install @storybook/addon-storyshots-puppeteer, it is an official addon to add automatic Snapshot Testing for Storybook.

yarn add @storybook/addon-storyshots-puppeteer puppeteer @storybook/addon-storyshots --save-dev

Follow the official guide, create a new file storyshots.spec.js.

// From https://www.npmjs.com/package/@storybook/addon-storyshots-puppeteer

import initStoryshots from '@storybook/addon-storyshots';
import { axeTest } from '@storybook/addon-storyshots-puppeteer';

initStoryshots({ suite: 'A11y checks', test: axeTest() });

Add a new command in package.json.

"test:a11y": "jest storyshot"

Now you can run storybook (yarn storybook) and run yarn test:a11y afterwards to test your settings. If your tests run correctly, you should see the test results.

For example, in my design-system-demo, I saw errors from two components. It also includes a link of how to solve it (example).

After the storyshots work locally, it's time to create a pipeline!

Create a pipeline

Warning: this section is long, make sure you have some knowledge on docker, docker-compose and CI.

Dockerfile

First, we need to run both storybook and storyshots in pipeline simultaneously. But one step in the pipeline only did one thing, how can we achieve that? Docker.

Create a docker file for storybook, and use docker-compose to create another service to run accessibility tests.

FROM node:14.16.0-alpine3.12

WORKDIR /opt/app/

COPY package.json ./
COPY yarn.lock ./

# Install app dependencies
RUN yarn install --frozen-lockfile
COPY . .

EXPOSE 6006

# Run storybook
CMD ["yarn", "storybook"]

Create a Dockerfile and .dockerignore file to ignore node_modules in your harddisk. This docker file is copying all files from local to docker image and runs the storybook in 6006 at the end.

docker-compose

Then docker-compose, create a file named docker-compose.yml:

version: "3"

services:
  web:
    volumes:
      - app-content:/opt/app/
    build: .
    logging:
      driver: none
    networks:
      defaults:
    ports:
      - "6006:6006"
    tty: false
  accessibility:
    image: "buildkite/puppeteer:7.1.0"
    networks:
      defaults:
    working_dir: /opt/app/
    volumes:
      - app-content:/opt/app/
    depends_on:
      - web
    command: /bin/sh -c "npx wait-on http://web:6006 && yarn test:a11y"
volumes:
  app-content:

networks:
  defaults:
    driver: bridge

There are two services here: web and accessibility. docker-compose runs two services simultaneously and accessibility wait till 6006 port of web is ready.

The takeaway of this:

networks setting is important, both needs to be in the same network, otherwise wait-on doesn't work
use puppeteer docker image in the accessibility service to save the time of setting headless Chrome and puppeteer
use logging: driver: none to get rid of noises from the web service

Then, change the storybookUrl of storyshots:

initStoryshots({
  suite: 'A11y checks',
  test: axeTest({ storybookUrl: 'http://web:6006' }),
});

Finally, the pipeline!

CircleCI

In your pipeline, add a job called accessibility.

  accessibility:
    docker: 
      - image: cimg/base:2021.03-20.04
    steps:
      - checkout
      - setup_remote_docker:
          version: 19.03.13
      - run:
          name: Accessibility tests
          command: |
            set -x
            docker-compose -f ./docker-compose.yml up --exit-code-from accessibility

The important stuff here: --exit-code-from <YOUR-DOCKER-SERVICE-NAME>.

By default, CircleCI gets exit code from the image itself, this means, even you have failed tests in accessibility, the pipeline is still passed.

By setting this exit-code-from, we can control the behaviour of pipeline to get exit code from docker container accessibility, i.e. when accessibility tests failed, pipeline throws an error.

See, the pipeline failed after setting exit-code-from!

One thing that is for CircleCi only, is the setup_remote_docker command (see official doc), which is connecting to remote docker service. Otherwise, docker-compose doesn't work without docker.

Add that to your workflow, for example:

workflows:
  tests:
    jobs:
      - test
      - accessibility

That's all. Enjoy!

Feel free to check my repo and my pull request for the a11y pipeline!

Beware, manual testing is still important!

Automatic test of accessibility is convenient but it can only catch a small number of accessibility issues.

Some sophisticated issues, e.g. a form that doesn't work well with VoiceOver, still need manual checks.

If you do care about accessibility, remember to do some manual checks!