Standing ground of VR today
Author : Olivier Lapointe
Date of writing: 2016 – 08 - 20
Pdf accessible here
Abstract
Virtual Reality is still very young. However, its coming, its here to stay, and it has very high
demands. Overall this paper is video game development oriented and gives a heads up
on what VR is, its limits, problems, costs and avenir. Its main focus is on the Head Mounted
Displays such as the oculus rift and the HTCVive. It also proposes a section introducing
many other VR devices. It goes further into details explaining what motion sickness is, what
causes it and how it can be addressed in video games. It offers a few methods of
locomotion used to fight motion sickness. It gives a summary and an analysis of the games
released in VR and the ones coming in the future. Lastly, it also approaches the
computing and hardware cost of developping for VR and its expectancy of progress for
the future.
Introduction
This paper talks about the constraints and requirements of Virtual Reality provided by
Head Mounted Displays in video games. VR is coming and aims to bring current
generation gaming up to amazing new heights. However, on the content development
side, this level-up is nowhere as simple to just duplicating the viewport and rendering
these with a slight shift on the yaw. Developping for VR is much more ressource expensive
than for traditional monitor gaming. This paper will begin by dwelving into the cost of VR
to give an idea of what to expect on the hardware side. Next it will introduce many of the
released and upcoming games for VR and provide an analysis of what they are, what
makes them special, as well as what is missing. Althought the main focus of this paper are
HMDs, it also introduces some of the other VR devices and how they could boost the
overall immersion. After that it’ll dig into the motion sickness problems of VR and what can
be done to mitigate or nullify these adverse effects. Lastly it’ll expand a bit more on the
subject of motion sickness and present a few methods of locomotion to be adopted and
adapted in VR experiences.
Computing Requirements for VR
This section talks about the cost of developing for VR; where GPU’s and screens stand
today and where they need to be going. To do so I will mention facts as well as attempt
to make simple calculations to give a middle ground where to land our expectations.
Let’s start this section by taking a big punch by looking at the raw amount of pixels we
need to render for VR with this slide from Alex Vlachos’s speech on Advanced VR
Rendering at GDC2015.
Alex Vlachos’s, Advanced VR Rendering, GDC2015 i
Doing some simple math we can deduce that VR requires 3 to 4 times more rendering
power than current monitor gaming, at the current HMD resolution. Though eye tracking
and FOV rendering will be coming later on, this is still a big jump. Even more when we take
into consideration that this resolution is nowhere near what we would want to render.
GPU’s up-to-date
Let’s take quick look at where GPUs stand in the consumer market today and how well
they’re performing. Current generation GPUs like the NVidia’s Gtx 980 perform perfectly
with today game demands, 1080p @ 60fps, everything maxed out. The PS4 and XBoxOne
come equipped with an equivalent GPU processing power to NVidia’s 750ti
ii
, which has
1/4 the performance of the 980, and still seems to perform pretty well at 1080p @ 60fps.
The following chart displays the increase of performance Nvidias has made with its GPUs
by comparing the latest Titan X Pascal with the top cards from their previous series.
Name
580
680
Titan Z
980
Titan X
1080
TitanX P
Release
Date
11-2010
03-2012
03-2014
07-2014
03-2015
05-2016
08-2016
Perf % to
the TXP
370%
250%
160%
100%
55%
20%
0%
Release dates from Wikipedia, and performance gains calculated with gpu.userbenchmark.com
Taking into account these simple numbers, we can observe GPUs almost double in
performance every 2 years and that we’d have to wait 4 more years for the ‘Gtx Titan X
Pascal 2’ before moving into ‘serious’ VR game development. However, these are just
simple and raw numbers to be taken into consideration rather than a live by rule as there
are many more factors to take into account. First of all comparing GPUs isn’t this simple,
as numbers can mean something on paper and something totally different in-game as
new technologies emerge. New technologies such as DX12, which don't change the
GPUs themselves, but how they process information. Secondly, and above all, it really
boils down to the type of game. A first person hyper-realist multiplayer fast paced game
will perform totally differently when compared to low-poly single player wall ping pong
game. Lastly we can just look at 2 already released incredible, beautifull and realistic
games; Adr1ft and EVE Valkyrie, and witness their excellent performance on the Gtx980.
In the end, all this means is that VR has higher performance demands than traditional
gaming, but great games can already be accomplished.
Head Mounted Displays up-to-date
On the HMD hardware side, the most important factors are the refresh rate, the
PixelsPerInch (ppi) and the quality of the screen. The minimal refresh rate on monitors and
TVs is 25fps, below this number our eyes can make up the jump between frames. However
in VR the screen display is right before our eyes and covers much more of our field of
view, this raises the minimal fps to 75. Anything below will break the immersion and even
make the user feel sick. Good news is; all current headsets already have a refresh rate
between 90 and 120fps.
Bad news is, since the display is up closer, the pixels on screen appear bigger as the PPI
isn’t enough. In game this translates to blurry objects, big broken edges and unreadable
texts. The Oculus Rift DK2 has a PPI of 386, its commercial version (CV1) and the HTC vive
have 455ppi. The PSVR will have 386ppi as well. These values are great, but they are
nowhere near minimal comfort zone.
From my own experience with the Rift DK2, I’d say that a PPI 3 times more significant
(1200ppi) might be enough to make each individual pixel not easily distinguishable.
Though for the ideal experience we might need to reach 8K displays, which is 8 times the
current 1200x1080 resolution per eye in those headsets. If my linear and optimist math is
correct, this would lead to 3000ppi displays. This is pretty ambitious on the hardware and
the processing power required to render such large frames.
All isn’t without hopes though, at least on the hardware aspect, as current market
available smartphones are already capable of 806ppi. Generally smartphones have a ppi
of 350, the GalaxyS6 reaches 577ppi and it is the Sony Xperia Z5 Premium that owns the
highground with 806ppi LCD. However, smartphones screens tend to be LCDs whereas all
VR HMDs require OLED screens. Different technologies mean different progress
breakthroughs. The Sony Xperia Z5 came out in November 2015, now, almost a year later,
Samsung prepares to release the Galaxy S8 with a 806ppi AMOLED screen. With this said,
it is important to remember that with VR mounts, such as the Samsung GearVR, it is
possible to use a smartphone’s screen for VR.
The requirements of tomorrow
So the future of HMD hardware is promising here too, but what about the GPUs? If we
can’t render such HD images, of what use are these HD displays? Though it may not fill the
whole gap, the aforementioned Field Of View rendering might help by reducing the load
we send to the GPU. With eye tracking we can know exactly where the user is looking at.
With this information we can decide what part of a scene to render in high quality while
setting all the rest to a lower one. This means that we wouldn’t be rendering the entire
screen at 100% but only ~20% of it and cap all the remaining 80% of the screen between
60% and 20% quality. There is no quality loss of image from the user’s perspective, but
there is no telling yet either how much work it would spare on our GPU.
Foveated Rendering, SMIvision talk iii
Considering all this new information on the ideal resolution we would dream the HDMs to
have, lets kick in some simple linear math to calculate how many Gtx Titan X Pascals we
would need for modern games to render in 16K, everything maxed out. By taking the
performance information of the Gtx TXP from eurogamer
iv
and applying linear math we
come up with 32 Gtx TXPs. Without FOV rendering and DX12/Vulkan.
Linear Optimist Math:
-Let’s take Far Cry Primal as the middle ground.
-At 1080p the Gtx TXP can render 260Million pixels per second (132fps * (1920*1080)).
-At 1440p it does 350M pixels per second (96fps * (2560*1440)).
-At 2160p 400M pixels per second. (50fps * (3830 * 2160)).
-To render at 16K we need to render 11 500 M pixels per second. (90fps * (16K*8K)).
-Taking the middle ground in Million pixels/second, we would need 32 Gtx TXPs.
Performance of the Gtx Titan X Pascal on various games, eurogamer.
Scary, right? However, if we consider that GPUs performance doubles every 2 years as it
did with the recent 980 and 1080, it might just take 10 years before reaching that point in
performance. We have to keep in mind that these stats are for intense ressource
demanding games and that many other great games don't need that much. Lets
remember that the workload might as well be cut in half with FOV rendering. In the end,
8K is an ideal resolution, 8K and 4K would already greaten the experience by a lot. Also,
we have to keep in mind that there are many other parameters to take into account.
What this information and non-relevant math do here is provide us an idea of where were
standing today, where we need to go and what to expect in the future. The other thing is,
we do not need to wait for all this gear to come before having great and awesome VR
games as.. there already are many of them out there !
Games for VR
While waiting for better GPUs and screens to release, let’s take a look at the awesome VR
games that are already here. I have compiled a list of games representing their genre.
Right after I will present a short analysis of what is out and what is missing. Most of these
games have already been released and only a few of them are to come end of 2016, or
2017 with the release of PlayStationVR. They can be found on Steam and the Oculus
Home store platforms. Other great free games and experiences can be found on WearVr
and itch.io.
Dragon front : tower defense game with top-down and ‘turret’ view.
Battlezone : battle tanks in arenas with first person cockpit view
Adr1ft: space station exploration in 1st person with emphasis on realism and the loneliness
of space.
Rigs: cockpit view, giant robots battle in arenas. Fast and agile gameplay.
EVE valkyrie: Mutiplayer space ship dogfighting in first person.
Gran turismo sport: car racing,
Classroom aquatic: exam cheating simulator in a dolphin school by making disctractions.
Eagle flight : bird flight simulator in Paris with friends.
Robinson the journey : 1st person exploration adventure in dino-land.
The climb : 1st person rock climbing and sight gazing.
Rock band vr : rockband in vr.
Gunjack: 1
st
person space turret defense against waves of ships.
Surgeon simulator : aliens surgery in 0 gravity, now more precise than ever.
Job simulator : mess around with office and kitchen objects.
Time Machine : Pod under-water and sky exploration with time traveling to record
different prehistoric species.
Vr Karts: Kart racing in vr.
Minecraft VR : minecraft in vr.
Resident evil 7 : horror shooter.
Paranormal Activity: horror/suspense game.
Farpoint : 1st person alien planet exploration /survival
Fallout 4 VR : game adaptation for VR.
CCP's Project Arena: tron like disk arena.
Serious Sam VR: 1
st
person frenetic shooter with beheaded kamikazes.
Star Trek Bridge: coop space simulator & ship management.
Chronos : 3rd person fixed cameras adventure .
Tekken 7 : fighting in 3
rd
person fixed camera.
Windlands : 1
st
person grappling hook exploration on floating islands and ruins.
Radial G : 1
st
person tube racer.
Subnautica: alien planet underwater exploration and survival.
Farlands : Planet exploration and discovery of alien life forms and behaviors.
Technolust : immersive 1
st
person cyberpunk adventure novel where you unravel mysteries
of a new world.
Budget Cut: Indoor exploration and puzzle solving with blink movement system.
Audio shield: space/tron like 1
st
person ping pong rhythm.
Brookhaven experiment: zombie onspot shooting.
Holopoint: vr archery with the Vive’s wands.
Fantastic contraptions: build crazy awesome vehicle & make it roll
Hover Junkers: Roomscale ships multiplayer fighting and shooting.
Selfie Tennis: Solo tennis in ‘unicorn land’.
From all this we can notice a lot of games where you are seated in a cockpit and move
with it. The main reason behind this is because it greatly reduces motion sickness.
Nevertheless, the experience is pleasant and the gameplay works very well. This is the best
way to start experiencing VR games. Some of them are: Time Machine, Hover Junkers,
EVE Valkyrie and Gran turismo, just to name a few of the vast majority.
Next, there are many ‘turret’ games where you stay on spot and do stuff but do not
change location by an inch with the controllers. This is also because of motion sickness, if
there is no cabin to fool the mind while moving, then user will probably get sick. Therefore:
no movement. Some of them are: Fantastic Contraptions, Gunjack, Brookhaven
Experiment Surgeon Simulator and Job Simulator.
Some games do overcome this limit a little bit by allowing the player teleport itself
somewhere else instantly. This method is 100% motion sickness proof and works wonders.
However, it isn’t very immersive. Some of them are: Budget Cuts, Farlands and Bullet Train.
Other games allow you to move freely on ground and rotate the camera as much as you
want. They can heavily induce motion sickness to non-experienced players but are a
breath of fresh air in freedom of movement for those it doesn’t affect. Some of them are:
Technolust, Adr1ft, Robinson the Journey and POLLEN.
More heavy games break the ground and allow the player to swing tree to tree or move
freely in the 3 directions. They would be expected to induce instant motion sickness.
However, in the case of Windlands I’ve only heard great opinions. And having personally
played Subnautica for more than 15hours, I can give an insight of my experience. The first
3-4 hours were harsh when rotating the camera with the stick. However, after the 8h mark
I’d only feel a minor discomfort probably due to the poor optimization of the game for VR,
and the low resolution of the DK2. By now I can dive in for 3h straight before having to
surface again, not because of feeling bad, but because of my way of playing. Of course,
adaptability is different for everyone.
Overall, all of these games are in first person view. Third person games and top-down
games such as Darksouls, Starcraft2 or Diablo3 are nowhere to be found. There aren’t any
frenetic first person shooters like Titanfall or COD either. There is however a ‘Super Mario 3D
world WiiU’ (3
rd
person platformer) like called Lucky’s Tale. No 3d or 2d Turn Based Game.
There is a flappy bird like 2d demo sidescroller called Bubble Inferno, but no real 2d
games. Tabletop games such as chess and Hearthstone are on their way
v
. We may also
soon be expecting bowling, pool, mini-golf and wack-a-mole type of arcade games
where you can interact with your hands like you would outside of VR.
As of today, still many common games types are missing in VR. This is not surprising
however, as VR still faces many problems; starting with motion sickness, the resource cost
is also higher, there aren’t enough devices out there for companies to want to support or
make such games, some genres would have no or almost no benefit of going VR, the
freedom of camera could break the rules for some, others would suffer from the display
resolutions, but above all, VR is still very young. Some of these games will come very soon,
others will take their time, and the rest will come much later on when HMDs will be
capable of truly replacing our monitors and TVs.
The many Other VR devices
When talking about VR we mostly think about the Oculus Rift, HTC Vive, google card
board and other Head Mounted Displays as they utilize our most important sense; vision.
Rarely do we ever think about all the other ways we have of interacting with the world.
This section will do just that; introduce the many other devices that along the HMDs will
greatly increase the sense of immersion in VR. As I won’t go too much into particular
details, I’ll often join links that lead to more information on how they work.
Head Mounted Displays : The Occulus Rift and HTCVive are the most known HMDs, but
there also are a few others to keep track of. The Razer OSVR HDK2 packs similar hardware
specifications and defines itself as the open source VR Headset. On the living room
console side, Sony has the PSVR headset, aka Morpheus, coming up soon.
With the immersion HDMs provide, we also want to have more immersive means of
interacting with the world around us. We’ll no longer want to simply press the ‘E’ key to
interact or pickup objects, we will want to do so physically or mentally and have physical
feedback of us doing it. This sections aims to introduce other upcoming technologies that
if don't become absolutely necessary with HMDs, will greatly enhance their experience.
Controllers
Having new ways of seeing and interpreting the universe around us, opens up the
possibility of new ways of interacting with it.
Sixsense Stem Controller : This controller takes your traditional wireless dual stick controller,
splits in half for each hand and adds 5 point positional trackers for your hands, feet and
head. This allows the user to perform basic leg and arm movement with their body and
see it reflected in-game. It works well enough for the user to see himself walking, jumping,
spinning and sword fighting. This controller having only 5 positional trackers relies heavily
on inverse kinematics to tell how each arm or leg is positioned and bended.
Full body test: https://www.youtube.com/watch?v=jkOLswJlTBs
TacticalHaptics’s ReactiveGrip : They basically took a typical stick controller and added
force feedback so the user may feel weight, gun recoils, sword cling, and other weapon
or hand wearable physical properties. On the demo it was used as an add-on to Stem’s
predecessor, the Razer Hydra.
Myo Armband : This armlet detects muscular activity on the arm and hand it is worn on. It
can only recognize up to 5 gestures (wave left, wave right, spread fingers, fist, and thumb-
to-pinky). Its application is somewhat limited to the patterns it can detect, how well it can
recognize them without triggering false positives and the use we can come up with. For
now Myo has already proved itself in its usability for presentations, video and music
players (think of next, previous and pause functionalities). However it also sees its
appliance in video-games. On their website, Myo provides support for many games such
as FruitNinja, Civilization5, NFS MostWanted, Minecraft and Skyrim, just to name a few, but
actual gameplay is yet to be seen.
In life Myo applications: https://www.youtube.com/watch?v=te1RBQQlHz4
LeapMotion : This little black box offers neat dual hand and fingers gesture recognition via
a camera. Yet, it is somewhat sensible and does not support hand over hand interactions.
It work really well as a mean to interact with your screens. Basically, what it does is add
touchscreen functionality to any screen (monitors, TVs, projectors). On the gaming side,
seeing and using your hands in-game is a huge plus and there already are many mini
games of the leapmotion website as well as experimental games on itch.io to try out.
Test LeapMotion: https://www.youtube.com/watch?v=bvkyEUb2NnA
Eyetracking
Eyetracking in VR enhances the experience by a lot; the ability of knowing where the user
is looking at the screen removes the need for tiring neck-movement and mouse usage. Its
main functionality is to facilitate interactions, but with FOV rendering it might also reduce
GPU workloads.
Fove : Is the first Head Mounted Display to feature integrated eye tracking. On the specs
side it has similar features to the Rift and HTCVive.
Eye Tribe Tracker : The Eye Tribe have come up with a stand-alone eye tracking device
that can work with any TV or integrated into other HMDs.
Tech Demo EyeTribe: https://www.youtube.com/watch?v=NZaQEQrk15A
Body Mocap
Sixsenses’s Stem provides basic body tracking, however PerceptionNeuron and PrioVR
take the next step by increasing the number of sensors from 5 to 12 minimum and up to
32. With that many trackers on the body, pose recognition becomes very easy and
reliable enough to even be used in mocap animations.
Perception Neuron test: https://www.youtube.com/watch?v=cvqL2LQjBgQ
Base platforms
One of the main limitations with VR is the limited space you’re playing in, which restricts
your body movements to basically standing, hoping to not unconsciously bump into a
wall or a lego.
Cyberith’s Virtualizer and KatVR’s movement platform allow the user to walk, run, crouch
and jump endlessly on one spot, without moving by a single inch.
GTA5 gameplay with the Virtualizer and Rift:
https://www.youtube.com/watch?v=6CZmJvI8mfc
Glove haptics
Gloveone and Dexmo have basically made mocap for hands. Their gloves allow tracking
and recognition much like the LeapMotion does, however it is much more accurate and
does not have the problem of one hand blocking the another. Their devices have
somewhat different bonus capacities. Such as haptic feedback that allows the user to
feel the rain, guitar cords and butterfly wings amongst others. As well as force feedback
to simulate your hand holding something, for example offering resistance when trying to
fully close your fist while holding a fishing pole.
Kickstarter Gloveone: https://www.youtube.com/watch?v=RAZidk_mPc8
Dexmo demo: https://www.youtube.com/watch?v=MBBQrGug_Go
Body Haptic
TeslaStudios came up with the TeslaSuit, a suit that covers your body and simulates
different feelings such as heat, cold, touch and impacts.
Trailer TeslaSuit: https://www.youtube.com/watch?v=RMBwb8megGE
Brain
Emotiv’s Epoc+ is a Brain – Computer headset interface that allows the recovery of data
from brain activity… as well as facial. With it you can move a box in VR 3d space just by
using your thoughts. The immediate down side is that it requires a very high amount of
concentration for it to work and is therefore slow and sluggish to react. This however did
not stop some people from using it to move real life miniature mechanical robots.
Developpers from the game Son of Nor also use it to swap and cast spells. Also to note, 4
years after the release of the device, there aren’t many applications or usages being
announced or teased. However this could and would be a very great addition, and
definitely the future of how we want to interact in VR
Mind controlled robot : https://youtu.be/nX741DZw8l4?t=5m7s
Son of Nor epoc introduction: https://www.youtube.com/watch?v=U0ML3YWn34w
Speech
Pronouncing words to perform actions is the step right before mind control. Tie actions to
words and have them triggered without having to use a physical gamepad button can
be faster, feel more immersive and allow this button to be mapped to a more important
command. Because in VR we’ll have much more things to interact with and do, and we’ll
want them to feel as real as what we see.
Elite Dangerous Astra (space sim): https://www.youtube.com/watch?v=DRVCkUN_Mq8
In Verbis Vertus (spell casting): https://www.youtube.com/watch?v=oBRiuAaBhNI
Head Mounted Displays act as catalysts for all these other devices. They are already
great by themselves, but they will only reach their full potential when paired with HMDs,
and vice-versa. Some of them are still in the early stages and the arrival of VR to the
consumer’s market will greatly increase and speed their development.
VR Motion Sickness [
vi
,
vii
,
viii
,
ix
]
Virtual reality motion sickness is the biggest frontline issue VR faces with new users and
media covering. It can be caused by many things and affects everyone on different
levels of intensity. It basically boils down to you feeling sick while wearing a HMD. The
symptoms can include: general discomfort, headache, stomach awareness, nausea,
vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy. These will
dissipate relatively fast after exiting VR and the person will get back to normal. Though the
main cause for this is due to a difference between what you’re seeing and what your
brain expects to see, there many other factors to consider:
Mistmatched motion: This is due to a discrepancy between how we are moving in VR
and how our brain expects us to do so. For example when walking down cliff, walking up
stairs or simply having the ground pulled from under our feet. Our brain expects to feel
something, and to not feel anything makes him think we’re sick. Movement speed can
also apply here, as our brains aren’t used to our body moving around 20mts/s. Jumping
high in the sky and landing without crouching can also feel weird. When we move in real
word our body receives tons of information besides the eyesight, in VR only our eyes
receive information of movement. It is this lack of other information that make us feel
weird. Lastly, it is important to know that this affects everyone at different levels and that
some people don't feel any discomfort whatsoever.
Motion parallax: As we move in the world we expect things close to us to move faster
than those far away, a mismatch in this information will cause discomfort. Our brain has its
own way of interpreting depth and distances. Tricking him into making farther objects get
near to us faster than close ones is something he isn’t used to. In other words; don't make
the mountain approach faster than the penny.
Field of view: While having a very high, or low, FOV on our monitors does not affect us in
any way. With VR, it is important to respect the one established by the HMD. Because the
HMD is very close to our eyes, fills most of our view and promotes a sense of immersion, our
brain expects to see stuff exactly like it would outside of VR.
Mismatched reality: This comes in when what we see feels real enough to be real but
isn’t. For example when having a mesh represent your hands in VR that looks real but fails
at having the right scale, finger lengths, position or rotation offsets match yours. The same
can be said with realistic animations and facial expressions that aren’t 100% accurate.
They look good, but the little thing that misses makes it feel uncomfortable. In these cases
it is better to fake all along and cancel any expectations our brain could have. This can
also apply to the height of the camera in-game, as a 2.10mt tall person is used to see
things differently than a 1.50mt one.
Level of detail: This is very much related to mismatched reality in the sense that we
expect things to look how they should, but this time it is due to how developers create
these things rather than the feeling itself. The big word here is Normal Maps. Normal maps
act as images that fool our perception of volume by playing with the way edges and
details are lit when we can’t afford to put real geometry. For instance, on a wooden door
we often trick the wood details with normal maps instead of using vertices. This works
great as long as the camera remains far enough. However the illusion breaks when it gets
to close. On monitors this would only look ‘cheap’, but in VR it’ll make us feel awkward as
we see depth but don't see really see it.
Color: Some games have increased contrasts in colors and intense lighting effects. In VR
this could increase simulation sickness. It is therefore recommenced to keep normal
contrasts that don't ‘burn’ our eyes as well as not using too many bright-white tones that
would blind us outside of VR as well.
Camera Effects: It’s strongly recommended to not use Head Bobbing, Depth of Field nor
Motion Blur as all these effects come override our natural way of seeing things.
Forced orientation: Force rotating the camera to look somewhere or allowing the
player to do so with the right stick, will cause motion sickness. This is due to a conflict
between what the eyes see and what the vestibular systems expects to feel.
There are many ways of impeding this problem:
1. Not allowing it, simply lock the orientation to the HMD.
2. Instant rotation.
3. Clipping the rotation; rotate of N degrees instantly.
4. Reducing the FOV while turning.
However this isn’t always a problem as some people don't feel uncomfortable while doing
it and others grow immune to it. Thought, forcing camera rotation without the user’s
consent, will always feel awful. If developers want the player to look somewhere, they
must motivate him to do so.
Acceleration: Also caused by a conflict with the vestibular system, acceleration will
cause motion sickness. However, this is reduced (and even nulled) if the speed is
constant. The direction in which we move (forward, backward, left/right strafing) also
influences differently. We are most confortable at linear speed going forward and user
controlled.
Time: Motion sickness comes progressively and the longer the user stays in VR, the more
intense the symptoms will become. Conversely, motion sickness will mitigate over time the
more the user spends time in VR and gets used to it.
Refresh rate: While a refresh rate of 25 images per second might suffice on monitors and
TVs to trick our brain into believing he’s seeing a continuous action, VR requires at least 75-
90 frames per second. The framerate must also be stable and not oscillate too frequently.
Lag: Actions performed in VR must immediately be reflected in front of our eyes. If there is
a split second between the moment we turn our head and the moment we show the
right image, we will feel sick.
Life: Finally, anything that would make someone sick in real life will most likely make her
sick in VR as well. Someone who feels vertigo, claustrophobia and/or other phobias
outside of VR, will feel the same in VR.
Experimenting game locomotion controls for VR
Having read and researched about different ways of dealing with locomotion and
motion sickness in VR, there are some I got to try and others I didnt. In order to try these
other methods I made a small project with Unreal4. What I did in this playground is
implement some of these methods and design a level in which to test them. For
disclaimers, because of time constraints, I havent implemented all of the methods I’ve
read about nor made a test level that fully exploit their characteristics. The reason for this
is because a method might very well work on a small test such as this, but will probably
feel totally different when confronted to real gameplay. The end purpose of this
playground is to simply try these locomotion methods to get a general feeling of how they
could work in-game.
UE4 VR Playground [
x
,
xi
]
In this little project I did two things: implement 7 methods of locomotion and design a level
in which to test them. The methods are split in first & third person cameras. All methods
share similar input controls but differ at their particularties, which often is how to deal with
rotating the camera. The input method can be changed on the fly by using the 1-7 keys
or the select button on an Xbox controller. At every change, a board is displayed to
inform the user of the changes in input.
In game board informing the inputs of the selected controller.
As follows, I’ll introduce the different methods implemented and what makes them work.
1 Clip Angle: Freely rotating the camera with the right stick is extremely nauseating and
probably the biggest problem when trying to have the freedom of monitor controls in VR.
However, instantly rotating the camera of N degrees at a time does not trigger motion
sickness. In this example the user can rotate the camera of 90 or 22.5 degrees at a time
by using the shoulder and trigger buttons. Although this method feels great in terms of
comfort, it doesn’t feel as immersive and locks a lot of buttons. Even though we can
rotate our body in real life and see it happened in-game, relying on buttons is much
easier as physically performing the action every time can result tiring.
2 Tunneling: Depending of the user, sometimes the simple fact of moving forward can
feel nauseating. The solution for this is to instantly transport the user where he wants to be.
There are 2 ways of doing so; blinking and tunneling. Blinking instantly transports the player
to the targeted area, whereas tunneling does so progressively providing a sense of
motion. The tunneling is done quickly enough so that the player notices being displaced
but doesn’t have the time to feel it. For very motion-sickness proof controls we could well
enough combine Tunneling with Clip Angle.
3 Clip Orient: This method is somewhere in between Clip Angle and free rotation. With
the right stick we get to tell our body in which direction to orient itself. We pull the stick in a
direction and on release our body faces this direction. For example; when we pull the
stick backwards, the camera rotates 180 to face backwards. If we repeat this action,
we’ll be facing our previous ‘ahead’.
The action of tilting and releasing the stick to instantly change our orientation does not
trigger motion sickness, feels a bit more immersive than Clip Angle and also is much
quicker to perform. However, it is quite sensible and demands some practice before
being capable of pulling the perfect angle.
4, 5, 6 Third Person: Very much to my surprise, I couldn’t find any game offering 3
rd
person view controls. Therefore this part relies entirely on my experiences, which ended up
begin very comfortable and immersive. The most nauseating experience in first person is
to turn the camera with the stick. However, in 3
rd
person the effect is greatly mitigated
and almost feels natural. At least on the Z axis, as on the others it blows your eyes as much
as in first person.
In 3
rd
person the camera can rotate relatively to two coordinates: itself or the character.
In my experience they both feel very good, but I have a slight preference with the
camera rotating around the character as it made me feel more connected.
In regard to walls and the camera pulling itself closer to the character, I did not feel any
discomfort.
7 Caged Rotate: The main difference between 1
st
and 3
rd
person cameras is that in 3
rd
you have the character to look at all the time. Pulling your focus onto something constant
reduces the discomfort while turning the camera. In Eagle Flight Ubisoft reduces the Field
of View by obscuring the edge of the eyes whenever the user uses the stick to turn
xii
. This
simulates a “TV” effect and reduces motion sickness
xiii
.
Althought I did not finish implementing this method, what I did do is lock stick rotation
whenever the user was turning his head in the opposite direction. So if your head is
rotating left, you can only use the stick to pivot left as well. If your head isnt moving too
much, then you may use the stick in any direction. In VR it is something to rotate the view
without having your head do the same thing. However it is something completely
different and worse having your head turn one way while pulling the stick the another.
For the playground, if we recap, with VR, the locomotion the actions that are most likely
to induce motion sickness are: walking, turning (on yourself or around a target), climbing
straight or spiral stairs, jumping, falling and feel vertigo. I designed the playground to be
open world with lots of vertigo inducing places.
I’ve uploaded a test video of this project on youtube, as well as the build for anyone to
try, on this address: https://www.youtube.com/watch?v=40lG9ZxhKd8
Two examples of ThirdPerson VR games
To conclude this paper, I wish to give a little better idea of what third person gameplay
could be in VR. I’ve uploaded two gameplay videos of some projects of mine.
The first, don't let me Die & Retry, is a project I made with friends for school during my M1
at l’ENJMIN. It is indoors, goal oriented and atmospheric.
https://www.youtube.com/watch?v=lgb7aOSV9bw
The second is sort of my UE4 playground that I simply partially ported to VR. It’s outdoors,
open world and adventure oriented.
https://www.youtube.com/watch?v=mbXq7agbrpM
Conclusion
As of today VR still has many challenges to address, both on user comfort and hardware
requirements. Thought the hardware is not yet at its ideal stage, great experiences have
already been created, many more are on the way and we can rest assured that it’s only
a matter of short time before it reaches perfection. Motion sickness and what it means for
traditional locomotion in games already has developers searching and finding different
solutions. These alternate methods are split between comfort and immersion. At the
moment it mostly falls back on the developers to solve these problems in their games.
However, the development of other VR devices might very well solve them easily and
even increase the immersion at the same time.
References
Although most of the research done for this paper comes from many different sources,
these few are the most prevailing:
- For further readings on VR techniques and tips from developpers, I strongly
recommend GDCVault.com for all the Game Developper Conferences they held
and made available on their website.
- For news related to VR, I can recommend visiting roadtovr.com and uploadvr.com.
- For elaborated personal insights: medium.com with a search for ‘vr’.
- Lastly, for free/paid games and experiences: wearvr.com and itch.io.
i
Advanced VR Rendering with Valve’s Alex Vlachos - GDC2015
https://www.youtube.com/watch?v=ya8vKZRBXdw
ii
No real source. Somewhere between the 660ti and 750. Just reddit comments.
iii
Foveated Rendering, RoadToVR http://www.roadtovr.com/a-pocket-guide-to-foveated-
rendering-from-smi/
iv
Nvidias Gtx Titan X Pascal review, eurogamer http://www.eurogamer.net/articles/digitalfoundry-
2016-nvidia-titan-x-pascal-review
v
Tabletop simulator vr trailer, BerserkGames Youtube
https://www.youtube.com/watch?v=KuUhWPUDpGo
vi
Wikipedia, VR Motion Sickness - Kolasinski, E. M. "Simulator sickness in virtual environments (ARI
1027)". www.dtic.mil. U.S. Army Research Institute for the Behavioral and Social Sciences. Retrieved
22 July 2014.
vii
Unreal 4 VR Best Practices,
https://docs.unrealengine.com/latest/INT/Platforms/VR/ContentSetup/
viii
Interaction Design in VR, Vales Lessons. Yasser Malaika – GDC2015
https://www.youtube.com/watch?v=_vQo0ApkAtI
ix
A look at the psychology of doing VR right. Kimberly Voll – VRGDC 2016
https://www.youtube.com/watch?v=-owQfn-iYQw
x
Interaction Design in VR, Vales Lessons. Yasser Malaika – GDC2015
https://www.youtube.com/watch?v=_vQo0ApkAtI
xi
A look at the psychology of doing VR right. Kimberly Voll – VRGDC 2016
https://www.youtube.com/watch?v=-owQfn-iYQw
xii
Eagle Fligh Ubisoft, tech demo of movement,
https://www.youtube.com/watch?v=XzAgKgBEEAc
xiii
Blacking out periphery to reduce motion sickness, http://arstechnica.com/gaming/2016/06/the-
cure-to-vr-sickness-might-come-from-blacking-out-the-periphery/
Special Thanks
Special thanks goes to l’ENJMIN for lending me a Rift DK2. Thank you!