Input Methods in XR: Voice Commands

Voice commands allow you to interact with digital objects and interfaces without using controllers.

People may be expecting an Alexa or Siri-like experience, but the technology isn’t quite there yet for Head-Mounted Displays as of this post.

Currently, the technology is at a point where it’s a good experience if your native language is supported, but difficult for those whose primary language isn’t supported. It’s also more of a challenge for those with speech impediments, whether permanent or temporary.

Natural language commands are also not where they need to be for an ideal experience since it’s difficult for the technology to problem solve with complex language and grammar. This means that for now, we’re limited to simple commands.

---

Pros

• People can complete tasks with a single command instead of having to navigate through menus.
• People can stay hands-free for other tasks.
• It can be a somewhat natural form of communication when done well.
• It works well for simple interactions.

Cons

• Commands may not work well in noisy environments.
• It could be inappropriate for designated quiet areas or public spaces.
• Depending on the technology, people may have to memorize a set of ambiguous commands.
• It doesn’t currently support more complex commands or interactions.
• You can’t correct mid-sentence or the command will fail. You have to think about what you’re going to say before you say it.
• It’s currently difficult for non-native speakers and those with accents or speech impediments.

---

TL;DR Do’s and Don’ts

Do

• Avoid ambiguous commands that are hard to memorize.
• Accompany voice commands with on-screen visual confirmation feedback.
• Use simple vocabulary that makes sense in context.
• Stay consistent with wording and resulting interactions throughout the experience.

Don’t

• Don’t use voice commands that are reserved for the hardware system as these could pull you out of the experience.
• Don’t design the experience to only work with voice commands, or without supplemental UI.
• Don’t use commands that sound similar when spoken.

---

Recommendations

• Try to follow Microsoft’s approach of “See it, Say it” by using the words in the UI as the commands.
• Use voice commands as a supplemental input instead of primary input so that people can still use the experience in various environments and situations.
• Test your experience with multiple languages and accents.

---

Examples to try

HoloLens

HoloLens uses the methodology of “See it, Say it.” Whatever is on the UI is what you say. This is the simplest way to interact with the lowest learning curve. If you have access to a HoloLens, give it a try.

Oculus Quest and Quest 2

I don’t have a Quest 2 to test the voice commands on that model (so I can’t speak to the experience on that headset), but I do have the original Quest. I also used earbuds with a built-in microphone when testing. Even though I have a quiet voice, the system picked up the commands when I spoke. At one point my voice went out mid-sentence (common problem for me), but it still was able to understand the command.

You double-click the Oculus button on your controller to activate voice commands. When you do this for the first time, a tutorial will activate, teaching you the voice commands and allowing you to look through a list of the types of things you can say.

You can also use the command word “Hey Facebook” to activate voice commands like you would with Google, Alexa or Siri. This is an option you can activate/deactivate in the system settings.

---

Learn more

• How to Design Voice User Interfaces by Ditte Mortensen from the Interaction Design Foundation. 
• Conversation Design by Google
• Voice Interaction UX by Nielsen Norman Group
• Voice Input by Microsoft Mixed Reality

---

If you enjoy these articles, consider supporting me on Patreon.

I’m an Immersive Tech UX Design Professional with over 22 years of experience designing for kiosks, websites, mobile apps and desktop software for many well-known and not-so-well-known companies.

I’m not speaking on behalf of or representing any company. These are my personal thoughts, experiences and opinions.

---