Interactions in VR

Immersive interactions

Part 1: Limits and potential of virtual interactions

To understand the opportunities and challenges, we first need to look at how users interact with their virtual environment. The hands are particularly important here. Whether via controllers or hand tracking, the hands play a central role. Both approaches require different design considerations.

Hand tracking makes it easier to get started as it directly reflects the natural movements of the hands. Controllers, on the other hand, offer more flexibility as they enable more complex functions through different buttons, such as opening an inventory at the touch of a button.

Every interaction requires careful consideration of the possibilities for translating and, if necessary, abstracting it. When other interaction methods such as gaze control or voice commands are added, the complexity of this task quickly becomes clear.

Translated with DeepL.com (free version)

Part 1: Limits and potential of virtual interactions

To understand the opportunities and challenges, we first need to look at how users interact with their virtual environment. The hands are particularly important here. Whether via controllers or hand tracking, the hands play a central role. Both approaches require different design considerations.

Hand tracking makes it easier to get started as it directly reflects the natural movements of the hands. Controllers, on the other hand, offer more flexibility as they enable more complex functions through different buttons, such as opening an inventory at the touch of a button.

Every interaction requires careful consideration of the possibilities for translating and, if necessary, abstracting it. When other interaction methods such as gaze control or voice commands are added, the complexity of this task quickly becomes clear.

The Challenges

Of course, the virtual world also has its limits—literally. Movement, in particular, presents a significant challenge for designers in VR. Essentially, there are three ways users can move within virtual reality:

• Room-Scale VR: Users can freely walk around, with their movements directly mirrored in the virtual world. This method often feels the most natural but requires a lot of space and tailored level design.
• Smooth/Continuous Locomotion: Here, the user moves in the direction they push the controller’s analog stick—similar to traditional first-person games. A major drawback is the potential for motion sickness, as visual movement doesn’t align with physical sensations, which can cause discomfort for many users.
• Teleportation: The user aims at a position with their hand and is instantly teleported there, without fluid motion. This avoids motion sickness but compromises realism and immersion.

As we can see, movement in VR is possible but often not as simple as in real life.

Another limitation of the virtual world is sensory perception. Smells and tastes cannot be implemented in VR, but even the tactile sensation of objects presents a challenge. Virtual objects lack weight and provide no resistance when grasped. There are approaches to compensate for this lack—such as inertia effects for heavy objects or adaptive vibrations—but even these cannot fully replicate an object’s real properties.

Moreover, everyday interactions often have to be adapted to function in VR. One example is gripping with both hands: since the user isn’t actually holding anything, their hands can continue to move freely, which can detract from immersion.

The Fascinating Possibilities

Despite these challenges, VR movement metaphors open up new possibilities that can even surpass reality. One example is teleportation: it may seem unrealistic, but it allows users to travel to any location within seconds—repeatedly and without any risk. In one moment, a user can stand in a pedestrian zone, and in the next, be looking down at the city from a skyscraper.

Many of the apparent disadvantages also bring advantages. Although virtual sensory feedback cannot fully match reality, through effects, highlights, and gamification elements, audiovisual perceptions can be intensified in ways that would never be possible in real life. This unique experience leaves a lasting impression.

Furthermore, VR makes the impossible possible. Users can closely observe unique, dangerous, or untouchable objects and interact with them in various ways. A prime example is our application "The First FIFA World Cup™." Here, players not only experience historical events up close, but they can even virtually hold the long-lost first World Cup trophy from decades ago.

Part 2: Thoughtful Design

Translating real-world interactions into the virtual world is more complex than it first appears. Some interactions can be directly transferred, some only with difficulty, and others can even be digitally enhanced. But how should this translation be approached? This is where UX designers come in. Through a meticulous design process consisting of three major steps, every interaction is carefully analyzed:

User Analysis

The foundation of any digital interaction is a thorough analysis of the end user. This analysis identifies whether the user has prior VR experience, what their expectations and challenges are, and what they aim to achieve with the application. For example: Is the user already familiar with VR? This information influences how detailed the introduction needs to be and how complex the interactions should be. While experienced users appreciate challenging interactions, beginners can quickly feel overwhelmed. An application designed “for everyone” often ends up serving no one.

Interaction Analysis

The next step involves examining how interactions occur in reality. Which elements can be directly transferred, where are there opportunities for improvement, and where does abstraction become necessary? All this, of course, from a perspective the user can understand. Often, aspects in VR need to be adjusted. For example, small, closely spaced buttons can be enlarged to avoid input errors and frustration. Adding positive feedback, such as sounds or rewards, can enhance the learning experience even beyond what is possible in real life.

Test, Test, Test

Now, theory is put into practice. Does the designed interaction metaphor work? The designer first creates various prototypes and tests them internally to gather initial insights and feedback. These early tests help make fundamental adjustments. Once the application reaches an advanced development stage, extensive user testing is conducted. At this stage, the test environment is carefully prepared to uncover any potential weaknesses in the application. A cross-functional team oversees the entire testing process, and the test users are selected to closely resemble the end users. The feedback collected is thoroughly documented and serves as the basis for further optimization of the application.

Conclusion

In summary, representing real interactions in VR is more complex than many might think. However, despite the challenges, it is almost always possible to recreate these interactions virtually—sometimes with adjustments, sometimes even enhancing them beyond reality. The opportunities presented by innovative VR solutions are crucial not only for the user experience but also for companies aiming to optimize their training and development processes. At NMY, we rely on customized VR applications to help organizations achieve their communication and learning goals efficiently. Let’s harness the potential of virtual reality together and take your projects to the next level.