Agile usability testing at IWMW 2016
‘Understanding Users; Managing Change; Delivering Services’ was the theme of IWMW 2016 (the Institutional Web Management Workshop) – and agile usability testing seemed to convey this message perfectly.
Neil Allison from the University of Edinburgh ran a workshop which focused on ways to improve the usability testing process, particularly in the post-testing, problem-solving stage. Our first task was to watch some usability testing examples and highlight any usability issues that appeared during the test.
It was clear from each test we watched that the website in question was not the easiest to use. Broken forms, missing information and poor accessibility were issues that all testers (and the master class group) picked up on.
Why is prioritizing usability issues important?
It was evident from the long list in front of me that to fix every issue would be costly in terms of time and resources, so we then listed the three most serious problems. This was a difficult task, because every problem seemed important – as builders of a website we want it to perform as well as we know it can.
Once the three most serious issues were chosen, we went around the room and put forward some of the issues we had noticed.
David Travis from Userfocus highlights the need for prioritization of usability problems:
“Having a standard process for defining severity means that you can be consistent in the way you assign severity and means that you provide the transparency needed for people to check your work.”
Both transparency and consistency fit snugly with the key tenants of Agile. This method of consolidating problems is less time consuming and more agile-friendly because:
- there’s no need for one person to watch each individual video and take meticulous notes
- there are no lengthy reports to prepare
- actions for problems are decided quickly after they have been spotted
- the shortened time of the overall process allows for this type of usability testing to fit more comfortably with sprint cycles and iterations.
After taking notes on each video, we all had long lists filled with highly precise issues that had been pointed out in the tests. The next step was to prioritise these issues.
How do you prioritize usability problems?
This is the easy bit. To prioritise tasks, we used Dr Travis’ prioritisation flow chart. If your team is used to MoSCoW’ing tasks, then Travis’ flow chart will be a breeze. All that you need to do is take one of the usability problems that came up and go through the flow chart with it in mind.
Working through this process in the workshop clarified what the real problems with the website were and made the task of fixing the issues less daunting. After all, for the website in question we had gone from a list of over 25 issues each, to less than ten issues overall, to 2 critical issues that took top priority and the others that were just ‘medium’ or ‘low’ priorities. It really helped separate the wheat from the chaff.
Similarly to MoSCoW, each severity level is categorised. In the same way that a ‘Must’ equates to a task that needs to be done in order for the deliverable to be a success, ‘Critical’ translates as a usability problem that will make some users unwilling or unable to complete a common task. This would be an urgent fix. Full explanations of the other severity levels can be found on Travis’ blog.
Where do solutions fit into this?
Once the problems have been categorised, only then can solutions be suggested. Allison stresses that it is only the usability issues themselves that are prioritised, not the solutions.
Actions should be based on three factors:
- Is the solution obvious?
- Is there an easy development solution?
- Is there an alternative to development?
Key Takeaways
The process Allison guided us through at the workshop was streamlined and efficient. It really all hinges on his straightforward guidelines:
- get the right people in a room – everyone with a stake in the product
- watch a small number of short sessions – Allison advises: “as many as you can fit into the time you have (so probably not very many)”
- prioritise the issues and create actions.
Resources
Neil Allison’s full write up of this process can be found on his usability blog, and the slides from the workshop are also available. My article ‘What is usability testing and why do we do it?’ highlights the digital communications team’s current usability testing process.
If you are a student or a member of staff who would like to participate in future usability testing sessions, please email [email protected]