Designing for Virtual Reality
At the start of last year I was told that our special investigation team at Stuff, was working on a project about New Zealand Defence Force’s involvement in Afghanistan looking closely at a particular battle in which two Kiwi soldiers died.
The team wanted to use Virtual Reality to enable people explore that moment of the battle and understand what it’s like to be there on the ground; to give them a deeper level of understanding of how things happened that day. I was quite excited to be part of it; this was going to be a New Zealand first.
I didn’t have any experience in VR, I was reading articles and watching lots of VR and 360 videos, but I knew VR had a massive potential in storytelling and it was still pure and untouched. I accepted taking a part in the project as a UX designer straight away.
As we stepped into the project it became clear to us we had to re-think how to engage the viewer in an immersive way. Using the power of VR, we wanted to tell the story from the first person perspective and engage the viewers.
When we first started the project every day was a new challenge. There were many late nights reading article after article about how to develop experiences in VR, how much people struggle and how it doesn’t have any existing rules, structure and information yet.
It was definitely in an experimental stage. We were literally coming to work every morning reading, learning from those readings and executing in the same day simultaneously. Having conversations with our developers who are familiar with the medium helped us a lot.
How to involve the user?
On the first day of planning we discussed how we wanted to engage viewers with this story. We also had to understand what the terrain looked like. Our director and editor Toby Longbottom has created the environment for us from -literally- card boxes and toy models.
We decided the best way would be to immerse the users within the entire story, allowing them to interact with the environment and stand next to the troops and experience how tragedy unfolded.
We had a lot of whiteboard discussions around how to execute the best experience for the viewer. Where will they stand? Do we let them move around or let them just focus on the story? What if they’re looking at the sky while something important is happening on the ground? How can we get their attention?
As we progressed things started to make sense. We learnt spatial cues, where you present a sound from any direction to draw viewers’ attention so they look to that direction. We learnt about visual cues you highlight a spot in the scene so viewers’ attention is drawn to that direction.
Put it on your face
Next thing we learnt was if you’re working on a VR project you actually have to use the VR pretty much every day. I think we tried most of the VR apps in the market. I used to sit at my desk and put a headset on and go through at least 5 VR apps in order to learn how it’s built, where does it fall short and how engaging it is.
Every time we learnt something new or had an awesome VR experience we took notes, shared the findings with the team.
UX/UI in Virtual Reality
I think this was the most challenging part for me. Creating an experience in a 360 world is hard. It requires a fair amount of time to create a new methodology to adapt your classic UI elements into a 3D environment. This might sound easy but you have to give up thinking on a 2D screen think in 3D space.
How small a typeface can be? Is it readable? How far can you place a button on the screen? How does the viewer select a button and how long do you wait to activate it? I was in a virtual world now so UI needed adjustment.
I used some traditional UX principles when building a feature or a product which was still valid for VR:
- An engaging and immersive experience
- A good design which is easy to digest and understand. Users don’t spend a lot of time figuring out what they’re looking at
- Bringing simplicity to a complex problem, create a trust between the user and the experience.
And also answer these critical UX questions:
- Who am I designing for, who is my audience?
- What problems are we trying to solve?
- What’s the success look like?
Asking these questions and following the traditional UX principles helped the team to understand what our goal is and what we’re trying to achieve. This helped us to stay on track and focus on the main task.
First demo release of the app. Name tags too big, terrain looks odd, menu button is too big and distracting.
First look of the map. Easy to get confused where you are, UI is confusing.
Tools
UX in VR requires a new way of thinking. But you can still apply traditional design tools, this can be sketching on a piece of paper, designing a low-fi prototype on Sketch or a simple animation in Principle.
It was important to visualise how we wanted things to look like and adapt these into this 3D world.
I also used other VR apps and different types of headsets for user testing: is the app engaging, is it easy to navigate, can the viewer use the headsets easily, is the headset comfortable?
It was hard to predict how the experience might feel. At the beginning my 2D designs and mockups didn’t work. I was struggling to adapt these mockups into a 3D world. How far is this object? Is it close enough to reach out and select this button? How do I activate the menu?
Then I was lucky enough to find this template to use in the Sketch app. This helped me a lot, now I was able to place buttons, icons and other UI elements on a 360 environment and see it straight away on the headset.
What I learnt from this is: once you have an idea you should deliver this as a quick prototype and test on a headset and share it with others so they can also experience it and give you constructive feedback.
Our developers used Unity to build The Valley app. Unity is a cross platform game engine which allows you to create 3D games and applications for desktop and mobile. Our dev team also had 3D artists to create the environment, texture, effects and animation to enhance the overall experience for the viewer.
Test, Test, Test and Test …
The last step of the project was to test the app with as many people as possible, internally and externally. This helped us to see if the experience was immersive and gave us a chance to make last minute changes.
An Afghan soldier holding the wrong rifle which is also used by the New Zealand soldiers.
Just before the app was ready to be submitted to the app store we showed it to an ex-soldier and he told us guns that the Afghan soldiers were using was wrong! This caused minor panic attack and 11th hour changes.
The Afghan soldier carrying the correct rifle, an AK-47
What’s next?
I’ve been lucky enough to work with such talented people and Fairfax has empowered us to explore this new medium and support us from day one.
I’m sure VR will bring amazing opportunities and open new doors for us. I’m very honoured and proud to be a part of this team and had a chance to tackle something crazy and fun with a great team.
I’d like to say a big thank you to the Stuff Circuit team for letting us be a part of this, and taking this journey with us to explore this new medium together.
It has been a pleasure working with them, I already look forward to our next VR project and change how we experience the world and shape the future of journalism in New Zealand.
Watch the documentary
Here’s the link to the interactive website where you can watch all the 6 episodes. And make sure you download the VR app from the App Store for iOS and Android.