UX Design for Emerging Technologies: AR, VR, and MR

Explore how UX design intersects with emerging technologies like AR, VR, and MR.

Emilyann Gachko
Emilyann Gachko
Apr 9, 2024
Min Read

As emerging technologies like Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) continue to evolve, they present unique opportunities and challenges for UX design. These immersive technologies offer new ways for users to interact with digital and physical spaces, requiring UX designers to rethink traditional design paradigms and consider new dimensions of user experience.

Before delving into UX design considerations, it's essential to distinguish between AR, VR, and MR.

Augmented Reality (AR) 

AR overlays digital content onto the real world, enhancing but not replacing the user's environment. In AR, context is king. Designing for AR requires an understanding of the user's physical environment and how to enhance it with digital information meaningfully.  Examples of AR use cases: 

  • Education: SkyView lets users point their mobile devices at the sky, identifying stars, constellations, and satellites. It enhances learning by providing contextual information based on the user's location and the sky's current state.
Screenshot of the AR-based app Skyview showcasing a constellation on a smartphone screen.
SkyView uses AR to visually guide users through locating stars, constellations, and more right on their smartphone.
  • Navigation: Google Maps Live View uses AR to overlay walking directions onto the real world, improving urban navigation by showing arrows and directions directly on the smartphone screen.
  • Retail: IKEA Place lets customers visualize furniture in their own space before buying. Users can see how various items fit in their homes, helping with decision-making.
Image of IKEA's AR furniture placement functionality.
IKEA's app lets users insert furniture directly into their home with AR tech, allowing for visualization of how an item will look in the space.

Virtual Reality (VR) 

VR immerses users in a completely digital environment, offering a step away from the physical world and enabling experiences that range from virtual travel to complex training simulations. Apple’s new VisionPro is a great example of VR (although the VisionPro has some AR features as well).

Designing for VR poses unique challenges, primarily around creating an immersive and navigable experience. Since VR environments are entirely digital, designers have the freedom to create complex worlds but must ensure these worlds are accessible and intuitive for users. Examples of VR include: 

  • Immersive Entertainment: Headsets like the Apple VisionPro and the Meta Oculus mark a significant leap in VR technology. Designers have the opportunity to redefine user engagement by creating environments that are not only visually stunning but also incredibly user-friendly. UX designers can push the boundaries of what's possible in VR, innovate in designing interfaces that are both intuitive and enriching, and set new standards for UX design in the virtual realm.
This is an ad for the Apple VisionPro.
An ad from Apple demonstrates how the VisionPro can be used to for immersive entertainment experiences.
  • Gaming: PlayStation VR (PSVR) is made specifically for gaming, and UX extends beyond traditional screen interactions, encompassing physical movements, spatial awareness, and multi-sensory feedback. For instance, users are immersed in a 360-degree environment, making traditional navigation cues obsolete. UX designers must create intuitive interfaces that guide users through virtual spaces, ensuring that interactions feel natural and effortless.
  • Healthcare: Osso VR provides surgical training through simulations, offering healthcare professionals a safe environment to practice procedures and refine their techniques. UX designers work to ensure that interactions within the VR environment feel natural and intuitive, enhancing the learner's sense of presence and immersion.

Mixed Reality (MR) 

MR merges elements of both AR and VR, allowing digital and physical objects to coexist and interact in real time, creating opportunities for more complex interactions and applications. MR requires UX designers to blend the digital and physical worlds in ways that feel natural and intuitive. The design challenge here is to create experiences that take advantage of both digital and physical elements without overwhelming the user. Some ways MR is being used: 

  • Design and Prototyping: Microsoft HoloLens allows engineers and designers to work with 3D holograms of their models as if they were real objects, enabling real-time collaboration and iteration without the need for physical prototypes.
An ad for Microsoft HoloLens.
A demonstration of how someone might use HoloLens to describe a model using MR.
  • Education: Magic Leap's collaboration with the Wacom technology company aims to transform how art and design are taught, by enabling students to interact with digital objects as if they were physical, blending creativity and technology.
  • Preservation: The Rome Reborn Project is a digital recreation of ancient Rome, allowing users to experience the city at the height of its development through MR, offering a tool for understanding history.
Screenshot of Rome Reborn project.
An image from the creation of the Rome Reborn project, allowing individuals to immerse themselves in the historical city.

Ethical Considerations and User Privacy

​​As UX designers explore the possibilities of AR, VR, and MR, ethical considerations and user privacy cannot be overlooked. These technologies often require access to sensitive data about the user's environment, movements, and interactions. 

It's the responsibility of designers, developers, and stakeholders to stay informed about these issues, engage with users and communities, and strive for ethical standards that prioritize user well-being, privacy, and respect. Below are some examples of ethical considerations with AR/VR/and MR design: 

  • User Consent and Transparency: Ensuring user consent and transparency about how data is collected, used, and stored is paramount. This includes clear communication about what personal information is being recorded, such as voice commands or environmental scans, and for what purposes. 
  • Mitigating Psychological Impacts: The immersive nature of AR, VR, and MR can have profound psychological effects on users, ranging from motion sickness to impacts on mental well-being. Designers should consider these aspects by providing clear guidelines for use, incorporating features to minimize discomfort, and designing content that promotes a positive and healthy user experience. 
  • Avoiding Bias and Ensuring Accessibility: Ensuring that AR, VR, and MR applications are accessible and inclusive to all users, including those with disabilities, is a critical ethical consideration. This involves designing experiences that accommodate diverse needs, avoiding bias in content and interaction design, and ensuring that the benefits of these technologies are available to a wide audience. 
  • Social and Cultural Sensitivity: Designers must also be aware of the social and cultural implications of their creations. This includes considering the impact of immersive content on different cultures, avoiding stereotypes, and ensuring that content is appropriate and respectful. Experiences should be developed in collaboration with cultural experts to ensure accuracy and respect. 

Looking Forward: The Future of UX in Immersive Technologies

The evolution of AR, VR, and MR technologies is opening up new frontiers for user experience design. This shift requires a multidisciplinary approach, blending skills from spatial design, psychology, and interaction design to create experiences that are not only technologically advanced but also deeply human-centered.

UX design for emerging technologies like AR, VR, and MR presents a fascinating blend of challenges and opportunities. By focusing on user-centric design principles and embracing the unique capabilities of each technology, designers can create immersive experiences that enhance the digital and physical worlds in meaningful ways. 

As we continue to explore these new dimensions of interaction, the potential for innovation is boundless, promising a future where technology enriches the human experience in ways we have yet to imagine.

Interested in learning more about the impact of UX? Check out these other articles:

Voice User Interface (VUI) Design Best Practices

The Impact of Emotional Design on User Engagement

Gamification in UX: Enhancing Engagement and Interaction

Launch a career in ux design with our top-rated program

Top Designers Use Data.

Gain confidence using product data to design better, justify design decisions, and win stakeholders. 6-week course for experienced UX designers.

Launch a career in ux design with our top-rated program

Top Designers Use Data.

Gain confidence using product data to design better, justify design decisions, and win stakeholders. 6-week course for experienced UX designers.