Going Off the Rails: The Making of Bullet Train Ryan Vance Senior Rendering Programmer, Epic Games UNREAL OPEN DAY 2016

1. Going Off the Rails:
The Making of Bullet Train
Ryan Vance
Senior Rendering Programmer, Epic Games

2. HOW DID WE GET HERE?

3. Elemental VR
Released at E3 2013
Our first foray into getting UE4 to
render stereoscopic images and VR
hardware support. We ported the
“Elemental” tech demo which debuted
at GDC that year.

4. Strategy VR
Released at CES 2014
Using the Crystal Cove prototype, we
explored scale and positional
movement in VR by adapting existing
game content.

5. Couch Knights
Released at GDC 2014
Using the DK2, we explored social
spaces in VR, as well as simple
gameplay.

6. Showdown
Released at Oculus Connect 2014
Designed to max out the new Nvidia
980 hardware, while running at 90Hz
on the new Crescent Bay hardware.
Content was adapted from the
“Samaritan” and “Infiltrator” demos.

7. Thief in the Shadows
Released at GDC 2015
Collaboration with Weta Digital’s
R&D team to understand how film
content could be adapted to real time
VR experiences.

8. What do all of our previous
efforts have in common?

9. What do all of our previous
efforts have in common?
Adapted Content.

10. We have something new in the
Oculus Touch, so we should create
something new on the content side.

11. Bullet Train
Released at Oculus Connect 2015
Our first foray into serious, only-
possible-in-VR gameplay using motion
controllers.

13. Design Considerations for Bullet Train:
● Raise the visual bar over “Showdown”
● …while running on the Oculus recommended
spec hardware.
● Interesting environment / architecture
● Motion Controls
● Test the engine, rendering improvements

14. So, what should we do?

15. Airports (RDU in Raleigh, NC)

16. Airports (RDU in Raleigh, NC)

17. European Train Stations

18. European Train Stations

19. Early Versions of the Environment

20. Early Versions of the Environment

21. Mix and Match!
While the majority of the assets were
custom-built for Bullet Train, we did buy
some assets, and cleaned them up for use
in the level
● Train Exterior
● Weapon Pack
● Character reskinned from “Infiltrator”

22. Looks good, now what should we
do here?

23. Option 1: Hogan’s Alley
A standard “shooting gallery” style gameplay
Fun, supports multiplayer easily, but staying in one place is
boring!

24. Option 2: Action Film!
Move through the world, taking out enemies.
Inspired by action movie “one-shots” like Oldboy and Hard
Boiled
Hard Boiled (1992) Oldboy (2003)

25. Option 2: Action Film!
The moving “one-shot” is a classic technique for action
sequences
For us, it solves the “movement problem,” because the camera is
moving through the action continuously.
Oldboy (2003)

26. Anatomy of a Chain
1. Player has two enemies to engage

27. Anatomy of a Chain
2. Player tosses weapon at enemy

28. Anatomy of a Chain
3. Player teleports to the downed
enemy, grabs gun out of the air

29. Anatomy of a Chain
4. Player uses the same gun to shoot
the next enemy

30. Anatomy of a Chain

31. Option 2: Action Film!
Advantages:
• Movement through the environment keeps
things interesting
• Constrained in terms of content, animation,
and AI scripting needs
• “Chaining” gameplay

32. What happened in practice?

33. When you’re stationary and
interacting with objects in front
of you in the virtual world, the
two trackers have line-of-sight
to your HMD and both
controllers. No problems!
Option 2: Action Film!

34. When the player is moving
past their targets, it’s easy to
over-rotate!
This causes occlusion of the
hands, and breaks tracking.
Option 2: Action Film!

35. Option 2: Action Film!
No good! But what did we learn?
• Movement keeps things fun!
• Like in “Showdown,” careful non-player
controlled movement is fun
– Start them moving right away, but fade in!
• We need to control both angle and engagement
distance

36. Well, at least we learned
something. What now?

37. Option 3: Magic!
Let’s take the best parts of the previous methods:
• Stationary Shooter: controlled distance, always
facing the action
• Action Film: movement around the environment,
exciting action

38. Option 3: Magic!
Since you’re already a disembodied set of hands,
why not let you teleport?
Gives us control where the player goes:
– Limits the content needed
– Controls the direction they’re facing
– Less overwhelming to players

39. Option 3: Magic!
Place teleporters so that the player always face the action.

40. Option 3: Magic!
It works!
We use trickery to keep the action in the center of the
map, so the player is always facing what’s most
important.
But, there were a few tricks we needed to add to make it
feel right…

41. Option 3: Magic!
“Teleport Blink”
• A brief fade to white cues the player
• Sound helps immensely!
• We left a trail behind to show you where you’ve
come from

42. Option 3: Magic!
Teleporter Selection
• Originally, players looked at what they wanted to
teleport to, but that doesn’t work in frantic combat
• Moved teleporter control to the hand, with snapping,
which lets players point at targets

43. But…how do we get that awesome
action movie feel? Teleporters aren’t
close to the enemies…

44. Option 3: Magic!
In order to preserve the
hand-to-hand engagement
distances, we added the
“stealth teleport.”
Allows you to “reach out and
punch someone,” as well as
some cool combos.
The Raid: Redemption(2011)

45. Option 3: Magic!
Teleporters were a nice solution, but they’re not without
their own issues.
• Re-orientation can still confuse players
• People still want to move around their “local space”
• Action doesn’t always keep in the center
• People simply forget they can do it.

46. So, now we know how we’re going to
get around, but what about gameplay?

47. We have guns, grenades, and rockets
stubbed in, so how do we make it awesome?

48. In short, we need to make you feel like a badass.
The Matrix (1999)

49. Failure is not an option!
Very few have used VR, and even less have
used motion controls.
● No death, only more badass
● Lots of invisible helpers

50. Anatomy of a Throw
Not many people are great at throwing
things in real life.
However, throwing something and
hitting your mark is really satisfying!

63. Anatomy of a Throw
● Take the view, and trace
the plausible targets
● Compute path based on
the player’s hand velocity
and angle
● Try a few different arcs to
get it to the destination
● Don’t make it obvious!

64. Rocket Throwing
The original idea was for players to grab the
rockets, aim them, and then let them fly.
Some people got this, other failed spectacularly.

70. This worked great when designers played the game!

71. This worked great when designers played the game!
But when other people played…

73. Most often, when players failed, they were
trying to throw the rocket overhand.
Solution: Let them throw rockets!

74. The Mechanics of Rocket Throwing
● Player grabs the rocket
● Average the velocity of the rocket for the
past few frames to get the “intent”
● Hysteresis is your friend (and a great word)
● Add visual flourishes to make it feel less
controlled

75. This got most of the people most of the way
there…
But some people still had some trouble hitting the
target. What do we do?

79. (Assist Delay)

80. “The longer you suck, the less you suck!”

81. IT LOOKS GOOD, BUT HOW DOES
IT ALL RUN?

82. Faked Reflections

83. With our highly reflective
environment, you need
reflections to make objects
feel grounded.
Faked Reflections

84. Let’s leverage an old trick!
Use underground proxy
geometry (in this case, a quad),
so it skews the same way as the
actual object.
Texture it with a capture of the
object, and then modify the
reflection using the surface
data from the Gbuffer.
Faked Reflections

85. While we pay a higher base cost for the GBuffer, it lets us
leverage all that information for tricks such as this.

86. The same technique works for characters, too!
Faked Reflections

87. In this case, we use two static
meshes attached to the feet as
our proxies.
In the vertex shader, we use
proxy position compared with
the actor position to line up the
proxy geometry to the view, to
allow for cheap “animation”
Faked Reflections

88. We then blended between two
captures of the leg, one
forward and one to the side, in
the vertex shader, based on the
angle between the leg and the
camera
Faked Reflections

89. Flattening Materials
Reflections on the enemy
character were now cheap, but
his material was most certainly
not!
Adapting a trick from mobile,
we “flattened” the materials by
baking multiple materials
down to just two.

90. Flattening Materials

91. Before: 5 materials, ~270 inst. / mat After: 2 materials, 77 inst. / mat
Flattening Materials

92. Flattening Materials

93. Technical Improvements and
Optimizations

94. WHAT HAPPENS WHEN CONTENT
OPTIMIZATION IS NOT ENOUGH?

95. Because we had ambitious visual goals, mere
content optimization wasn’t enough.
Time to implement some general optimizations
at the engine level.
• Hidden (and Visible!) Area Meshes
• Instanced Stereo Rendering

96. Hidden and Visible Area Meshes
There’s a lot of wasted space that you can’t see!

97. Hidden and Visible Area Meshes
Hidden area meshes were
already covered by Valve’s
Alex Vlachos at GDC
2015.
Here’s how it works…

98. Hidden and Visible Area Meshes
Draw the hidden area mesh at the near plane before the depth pre-
pass and the base pass

99. Hidden and Visible Area Meshes
Next, draw the scene as normal. The GPU will reject pixels
already masked out by the mesh, because they depth fail.

100. Unfortunately, we weren’t
seeing the 10 – 14% gains
that others have seen by
doing that.
But, we pay a lot in post-
processing time!
Hidden and Visible Area Meshes

101. Use the inverse of the hidden area mesh, and use that for post
processing, instead of a quad.
Hidden and Visible Area Meshes

102. Much better! With the
two techniques combined,
we were seeing:
Oculus ~0.25 - 0.3 ms
Vive ~0.3 - 0.4 ms
PSVR ~1.0 ms
Hidden and Visible Area Meshes

103. Instanced Stereo Rendering
In our old way of stereo rendering, we’d duplicate every draw call,
which is easy to implement, but really taxes the CPU

104. Timothy Wilson proposed a
better technique at SVVR in
2015. Instead of duplicating
everything, let the hardware do
the instancing for us. Each
draw call draws both eyes at
the same time. We save CPU
time!
Instanced Stereo Rendering

105. By using instancing, we make one draw call draw two times, using
the left and right eye transforms.
Instanced Stereo Rendering

106. To see how this works step-by-step, let’s draw this simple test scene
using instancing.
Instanced Stereo Rendering

107. Draw geometry twice using instancing. One instance gets the left eye
uniforms, while the other gets the right eye uniforms.
Instanced Stereo Rendering

108. Next, since we’re drawing to half the view port, scale things by half
horizontally.
Instanced Stereo Rendering

109. Then, translate each eye so that they’re in the proper left and right eye
positions.
Instanced Stereo Rendering

110. Finally, clip the left and right eyes at the center border using user
clipping planes.
Instanced Stereo Rendering

111. This was a huge win! On Bullet Train, this saved
about 1.5 ms on the CPU, and 0.75 ms on the GPU!
Showdown had about 500 unique draw calls, where
Bullet Train had ~2300!
Instanced Stereo Rendering

112. Special Thanks:
Nick Whiting, Nick Donaldson, Chad Taylor,
Ryan Brucks, Jerome Platteaux, Tommy
Jacob, Artem Bolgar, and the rest of the
Bullet Train team!
Thanks!