I’m a big fan of streamlined processes and systems, so of course, I wanted to set one in place for usability testing on my VR projects. I won’t go into all of the pros and cons of research sprints -vs- fully-developed plans in this article. However, I will say that in my own experience most teams and projects don’t have the time or budget for robust research plans.
Even so, you will still want to conduct some amount of research with people in your target audience to help ensure your product is easy and effective. If you have a streamlined process in place for that, you’ll have a better argument for your stakeholders and money-holders to include research in the project. A research sprint is a proven, effective way of doing that.
Google Ventures did all of the hard work of documenting and setting up how the sprints work, so I won’t reinvent the wheel here. However, I have modified it to account for VR needs. So, the Usability Testing Sprint for VR that I use for my projects is based on the Google Ventures Research Sprint, but has been modified to account for some VR needs and is specifically for usability testing. Also, it will take a minimum of 5 days to complete instead of GV’s documented 4 days.
It’s not necessary for them to be consecutive days. They can be spread out over the calendar, but should not be shortened to less than 5 days unless you enjoy working very long hours.
As with my previous article, VR: A Different World, A Different Way of Testing, this article will focus on usability testing for VR applications specifically since Augmented and Mixed Reality testing would be best conducted in the field if at all possible. That would require more logistics than you may be able to accommodate within a 5-day sprint.
However, if testing in the field is not possible, this sprint could be used if you want to simulate field testing for AR or MR in VR.
The modified process
Again, it’s not necessary for them to be consecutive days. They can be spread out over the calendar, but should not be shortened to less than 5 days.
However, if you have a set of templates in place for your entire process including standardized interview questions and report synthesis templates, you may still be able to shorten some of the upfront work and maybe halve the 5th day. The full day of testing is still needed and should not be shortened.
5 essential components
The essential components are the same as with GV’s sprint. However, the 5th component is a rapid summarization of findings instead of real-time.
- A set of tasks, questions and assumptions. Nothing has changed here. You still need to decide up front amongst your team what will be tested, what information you will need to gather from the target participants, and what assumptions about the people and the solution you want to validate.
- Intentional and selective recruiting. This is important for any project, but there are some additional aspects you’ll want to screen for when testing a VR solution. As mentioned in my previous article, you’ll want to include questions regarding age range, and gaming and VR experience.
- A realistic prototype you can test. Regardless of the fidelity of you prototype or solution, you will want at the very least to ensure the performance is adequate enough to not cause sickness or discomfort. If they can’t finish the exercise, you’re not going to get information beyond the verification that prevention of motion sickness is vital to the success of your project.
- Five 1-on-1 interviews combining broad discovery questions with task-based evaluation of a prototype. GV’s article explains why 5 is all you need for a sprint, so I won’t go into that again here. As mentioned in the previous article, I would add on to this the pre, mid, and post-test questions to determine the physical and neurological effects of the simulation on the target audience.
- Rapid summarization of findings. The reason this is rapid instead of real-time is the added time it takes between tests to reset and wipe down the equipment for sanitation purposes. Due to the newness of VR, it’s also common for the test to go over a little so 1.5-hour time slots tend to work better than 1-hour slots. This means less time on the day of to debrief.
I should add the caveat that due to the nature of the companies I work with and how their development process works, the frequent small sprints aren’t normally practical so we’ve gone with larger chunks of solution to test. If you have a team and business dynamic that allows for the smaller, more frequent chunks as GV suggests, by all means do so. That could cut your time back down to the 4-day sprint as they intended.
A summary breakdown of each day
You can get more in-depth information from the GV library on how the days are broken down, so this will focus on how I break them out for VR.
Day 1
- Set the deadline for test day based on availability of core team and location
- Determine the location, and if necessary, begin location logistics
- Start recruiting participants
- Screening the participants is really important
Day 2
- Select five participants and two floaters from the screener results
- Schedule participants
- Draft interview guide or test script
Day 3
- Finalize schedule and logistics
- Confirm participants
- Complete interview guide and test script
Day 4
- Set up the room
- Set up devices and recording system
- Set up observer space in another room ideally
- Usability testing and interviews
- Summarize findings after each test
Day 5
- Generate reports from digital survey
- Create research reports
- Plan next steps
Be sure to give good instructions and sample introductory invites to the person recruiting participants so that the participants have clear expectations of what will happen during the study.
Consider creating a set of templates for usability testing scripts and surveys, introductory invite emails, rules of engagement, rapid debriefs and final reports.
Learn more
- The GV research sprint: a 4-day process for answering important startup questions by Google Ventures
If you enjoy these articles, consider supporting me on Patreon.
I’m an Immersive Tech UX Design Professional with over 22 years of experience designing for kiosks, websites, mobile apps and desktop software for many well-known and not-so-well-known companies.
I’m not speaking on behalf of or representing any company. These are my personal thoughts, experiences and opinions.
