Designing Across XR Technologies and Platforms

When designing a solution that is going to be used across multiple technologies and platforms, such as VR and 2D-mode on desktop, the translation between those platforms should be well-thought-out.

It is important to ensure that the controller mapping and other interaction elements are platform-specific and that UX best practices are followed for each, in order to create the best possible experience for the user.

TL;DR Do’s and Don’ts

Do

Match the onboarding experience to the appropriate platform.
Ensure the onboarding walkthrough matches the specific platform.
Take advantage of the strengths, and counter the weaknesses of each target platform.

Don’t

Don’t design a cookie cutter UI or onboarding tutorial and plop it onto each platform with little or no thought to the nuances of each input type.
Don’t take an existing 2D experience and convert it to XR with little or no thought to the user experience, performance, latency, controller inputs, onboarding, physical factors, motion, or legibility requirements necessary for a comfortable and compelling experience in XR.

Recommendations

Controllers

Input controllers should be appropriately mapped for each platform and technology that will be used for the application.
For example, if the final application will be available on desktop, VIVE and Oculus, make sure the appropriate controller actions are mapped for each and that the onboarding and tooltips are correct for each.

In-world instructions and training

Ensure that task instructions are appropriate for the platform and technology in use.
For example, the control on a console game may be to tap the ‘X’ button to perform a task, whereas the control in VR might be to pull the trigger.

UI elements

Font sizes and elements should be appropriate for legibility on each platform and technology.
UI elements should be consistent with user expectations for that platform and technology.
For example, click sizes for desktop applications don’t have to be as large as target areas in XR environments due to the more precise input methods available on the desktop. That said, it doesn’t hurt to have larger target areas on the desktop when possible to account for rapidity of actions and accessibility for motor function impairments.

Example case study

Name withheld so as not to defame the maker

There is a VR game experience on the market that was originally created for desktop and console for use with a game controller. When converting it into a VR game later, they did not optimize it for virtual reality. As a result, the interactions in VR are difficult to figure out.

Also, many of the best practices for VR experiences were not implemented, resulting in confusion and the inability to successfully play the game.

Specifically…

The user has to figure out how to use the VR controllers through trial and error since all in-game instructions reference a traditional game controller — not the VR controllers being used.
The desktop menu navigation was dropped into the VR experience without adapting the interactions for use with 6-dof controllers, making it very difficult to interact with the menu. When I finally did manage to make a menu selection, I didn’t know how I had done it.
The experience was not optimized to reduce latency resulting in long wait times and feelings of vertigo.
The reticle, which was necessary in the 2D version of the game, became a distraction and broke immersion within the VR experience.

Example to try for free

YouTube player — Rec Room is available in 2D and VR modes.

Rec Room VR

This social VR experience is cross-platform, including Playstation VR, Oculus Rift and Quest, and Steam VR-supported headsets such as the the Valve Index and HTC Vive. It will also work in 2D mode with iOS and Playstation 4.

When trying the experience across the various platforms to which you have access, pay attention to how the tooltips and controls were changed to match the platform and technology. Think about what works and doesn’t work as this will help you discover things to look out for when designing or assessing these experiences for yourself.

Learn more

A good example of considering the capabilities across platforms is to take a look at the evolution of the HoloLens and Windows Mixed Reality. This documentation has a chart explaining what is and is not supported across each of the headsets in regards to coordinate systems and experience scales.

Each of these strengths and weaknesses can greatly affect the experience.

Coordinate Systems – Mixed Reality by Microsoft

If you enjoy these articles, consider supporting me on Patreon.

I’m an Immersive Tech UX Design Professional with over 22 years of experience designing for kiosks, websites, mobile apps and desktop software for many well-known and not-so-well-known companies.

I’m not speaking on behalf of or representing any company. These are my personal thoughts, experiences and opinions.