top of page

A11yFirst User Study Write-Up

​

1) Anonymize User

​

My user will be called “D.”

​

2) Background of User

​

My user for this test was a middle-aged man who is very knowledgable about computers. He works for a business that uses WordPress, so he also has some knowledge about authoring using WordPress plugins. I choose him partly because he would be an excellent candidate to use the A11yFirst plugin in the future.

​

3) Details of tasks, including the exact wording you gave to your user

​

The user test was completed using the following website: https://a11yfirst.library.illinois.edu/plugins-dev/custom/index.html

 

The questions were as follows:

  1. Please write up a few lines of your choice. It should include a title for the page and at least one paragraph. The paragraph should be titled.

  2. Please edit the look and feel of the document so that you're happy with the design

  3. Please add a photo to your webpage

  4. Can you please check the accessibility of the document?

 

4) What they did

​

Before starting the test, I gave D. some background on A11yFirst by telling him that it is specifically an accessibility plugin. I choose to do this because my feeling was that someone using this particular plugin would be likely to know that it is meant to aid in authoring accessible documents. However, I do think that this may have led to D. being able to complete the tasks with relative ease. That is likely a good sign for the program, though I do not know if the results would be the same if I made a different choice.

 

For question 1, D. started typing up a title for the website, title for the paragraph. He left everything the same size. When I prompted him to see if he could make it look like a title, he went directly to the header button. I believe this may be due to his previous familiarity with WordPress. He was impressed with the fact that the program forced him into Header 2 for his paragraph title.

​
When I asked question 2, D. started exploring the plugin a little more. He tried to tab through the buttons since he doesn’t like using the trackpad, and discovered quickly that he was unable to. He tried ctrl tab and shift tab and still couldn’t do it, which annoyed him. Finally, he made some sections bold and looked for a color button - when he realized there wasn’t one, he was surprised.
3. D. ran into a few more snags when trying to add a photo to the document. First, he wanted to upload one that was saved on the computer and didn’t know why that functionality didn’t exist. Then he started looking around the add-image box and he noticed the text in the preview box. He didn’t want it and tried to delete it, but it couldn’t be deleted which was very frustrating. He deleted out of the add-image box and looked around for something else. He wound up clicking the accessibility button since he didn’t know what it was. Finally, he went to another website and took the URL of a photo and went back to the add-image box. He was happy that him image showed up in the preview box, but he noticed the original text was still below it. He was happy that the image without the text showed up when he went back to the main page, but he was mad that it was there in the first place. He did put words in the alternative text box immediately, so he was not stopped by an accessibility issue as he went back to the main page.
4. He was able to check the accessibility of the document because he had noticed and clicked on the box during question 3. I asked him if he would have known what that box was otherwise and the answer was no - he said “I had no idea what that symbol meant.” His document had no accessibility issues.

Since we still had a little bit of time after these questions, I asked him to choose a webpage and copy it into the plugin in order to check the accessibility. He chose an article from CNN. He copied that in, clicked the check accessibility button, and was able to fix the problems with ease.

 

After that, I asked him what he thought about the plugin. He noticed the lack of spell check and context check, which really annoyed him. He thought the plugin should use a word editor instead of an RTF editor. He also asked why he couldn’t change the color, and I explained that it is an accessibility issue.

​

5) What is interesting

​

I thought it was interesting that D. had a relatively easy time completing all the tasks. The lack of certain functionality (primarily color) didn’t faze him for long. I am concerned that by giving him the heads up that the editor was accessibility focused, I didn’t capture every problem a novice user might experience. However, I do think the editor should explain itself as being accessibility focused, so I think I gave him the information that all users will have.

​

6) What they didn't do

​

He didn’t try to change the text size when I asked him to make the document have a title - I assumed that it would be his first move, but instead he clicked heading. I think perhaps this question could have been phrased differently, though the phrasing didn’t contain the word heading. He actually never tried to change the text size at all throughout the study.

​

7) What was surprising to you

 

How easy it was for him. This may bode well for the usability of the program - there are certainly some things to be tweaked, but it doesn’t really look like there are major problems, or at least not ones that this test caught. I’m actually a little worried!

​

8) Candidate reasons why you think they did and didn't do certain things it

​

I think that D. knew to do the header button for the title because of his familiarity with WordPress. Since he’s worked on writing websites, he knows that titles are different from just larger-font words. Someone more new to it likely would not.

​

9) Relation to the readings

 

I think in this case, I really saw the effect of the prior knowledge you give the user. I think the test would have been different if I had started him with no information, but I’m not sure if it would have been more honest. I also thought it was interesting and important to do the test with someone who knows WordPress. Many users won’t and might stumble into problems that D. did not have to deal with, but we can now be somewhat confident that knowledgeable users can navigate A11yFirst. I guess this is why you try to get a diverse group of users!

​

10) Redesign Speculations

 

Items that need to be fixed:

  • Tabbing - especially in a program built for accessibility, users need to be able to tab through the buttons.

  • The accessibility button should either be redesigned or explained

  • Spell check should be implemented

  • There should not be any text in the preview box in the add-image box

  • The plugin should maybe contain some info about accessibility at the top

 

However, this user study did not yield any ideas for major design overhauls.

bottom of page