The document discusses real world virtual reality. It describes how virtual reality can be used to create unified educational experiences for classrooms using technologies like Google Expeditions. Key points covered include how multicast DNS and DNS service discovery allow devices to discover each other over local networks without internet access. Image capture and processing techniques like photospheres, optical flow and photo stitching are also summarized in regards to creating virtual reality content from real world locations and scenes.

1. Real World Virtual Reality
Alex Kesling

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
google-expeditions-virtual-reality

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. @AlexKesling
Alex.Kesling@gmail.com
http://linkedin.com/in/alexkesling

5. Ref [1]

6. Ref [2]

8. Modern CS
in the Real World

9. Modern
Computer
Science
The
Real
World

10. Modern
Computer
Science
The
Real
World

11. Modern
Computer
Science
The
Real
World

12. CS + Reality
= Software Engineering

13. CS + Reality
= Software Engineering

14. Modern CS
in the Real World

15. Real World Virtual Reality

16. What we will be covering?

18. Ref [4]

19. Ref [5]

20. Zeroconf Networking DNS-SD Interpupillary Distance (IPD)
AppleTalk Panorama Field of View (FOV)
NBP Photosphere Optical Flow
mDNS Omnidirectional Stereo Photo Stitching

21. Real World Virtual Reality

23. Ref [6]

24. Ref [7]

25. Ref [8]

27. Ref [9]

28. Ref [6]

29. So how do we do this?

30. Meet the Classroom

31. Ref [9]

32. Meet the Users

33. Ref [10]

34. Often No Internet

35. One Teacher + Thirty Students

36. Existing Social Dynamics

37. Schools are Cost Sensitive

38. The Problem

39. One Unified Experience

40. 31 Devices

41. Guide
Explorers

43. What options are available now
● Service Location Protocol (SLP)
● NetBIOS
● Web Services Dynamic Discovery (WS-Discovery)
● Universal Plug and Play (UPnP) / SSDP
● mDNS + DNS-SD (Bonjour)

44. Our Choice...

45. Our Choice = mDNS + DNS-SD

46. Some Context
Foundational Events Through Time

47. Some Context
1974
- TC
P/IP
Spec

48. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP

49. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- D
N
S

50. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S

51. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S
1994
- TC
P
stack
default included
in
M
ac
O
S

52. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S
1994
- TC
P
stack
default included
in
M
ac
O
S
1999
- IETF
Zeroconf W
orking
G
roup

53. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S
1994
- TC
P
stack
default included
in
M
ac
O
S
1999
- IETF
Zeroconf W
orking
G
roup
2000
- m
D
N
S
draft proposal

54. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S
2002
- Bonjour née
R
endezvous
1994
- TC
P
stack
default included
in
M
ac
O
S
1999
- IETF
Zeroconf W
orking
G
roup
2000
- m
D
N
S
draft proposal

55. Some Context
1974
- TC
P/IP
Spec
1982
- U
S
D
oD
Standardizes
on
TC
P/IP
1985
- AppleTalk
1985
- D
N
S
2002
- Bonjour née
R
endezvous
2013
- m
D
N
S
R
FC
2013
- D
N
S-SD
R
FC
1994
- TC
P
stack
default included
in
M
ac
O
S
1999
- IETF
Zeroconf W
orking
G
roup
2000
- m
D
N
S
draft proposal

56. Zero Configuration Networking

57. Name Binding Protocol (NBP)

58. Bonjour

59. mDNS + DNS-SD

60. So how does it work?

61. mDNS = Multicast Domain Name System

62. mDNS = Multicast Domain Name System

63. .local

64. mDNS = Multicast Domain Name System

65. mDNS = Multicast Domain Name System

66. Unicast
Ref [11]

67. Unicast Broadcast
Ref [11]

68. Unicast Multicast Broadcast
Ref [11]

69. 224.0.0.251:5353 (IPv4) or FF02::FB:5353 (IPv6)

70. One-shot Querying in mDNS

71. Responding to mDNS Queries

72. Continuous Querying in mDNS

73. DNS-SD = DNS Service Discovery

74. DNS SRV Records
# _service._proto.name. TTL class rr priority weight port target.
_http._tcp.infoq.com. 86400 IN SRV 10 60 5060 bigbox.infoq.com.
_http._tcp.infoq.com. 86400 IN SRV 10 20 5060 smallbox1.infoq.com.
_http._tcp.infoq.com. 86400 IN SRV 10 20 5060 smallbox2.infoq.com.
_http._tcp.infoq.com. 86400 IN SRV 20 0 5060 backupbox.infoq.com.

75. DNS TXT Records
# name. TTL class rr text.
infoq.com. 86400 IN TXT “QCon NYC Date=2017/06/26” “Number of Tracks=7”

76. DNS PTR Records
# name. TTL class rr text.
100.0.168.192.in-addr.arpa. 6230 IN PTR mailserver.infoq.com.

77. DNS PTR Records
# name. TTL class rr text.
_googexpeditions._tcp.local 6230 IN PTR Leif Erikson.local
_googexpeditions._tcp.local 6230 IN PTR Christopher Columbus.local

78. Leif Erikson
.local
Christopher
Columbus
.local
Explorers

79. Many Services use mDNS/DNS-SD
Expeditions: _googexpeditions._tcp & _googexpeditions._udp
Chromecast: _googlecast._tcp
Spotify Connect: _spotify-connect._tcp
Philips Hue: Philips-hue.local, but resolves with _hap._tcp

80. Leif Erikson
.local
Explorers

82. Lead Explore

83. Lead Explore
?

84. ?

86. Browse /
Download
Lead

87. Google Cloud
Storage
(GCS)
Serve
Browse /
Download
Lead Explore
?

88. Left: Ref [11], Right: Ref [12]

89. Ref [13]

90. Ref [3]

91. Ref [15]

92. Ref [16]

93. Ref [17]

94. Cylindrical Panorama

98. Projection

99. Cube Sphere

100. Ref [17]

101. Viewer’s Perspective
Ref [17]

106. Ref [a]

108. Ref [17]

109. Ref [18]

111. Ref [15]

112. Ref [12]

113. Ref [b]

115. Perspective Projection

117. Spherical Panorama

118. Spherical Panoramas = Photospheres

119. Ref [c]

121. Ref [d]

122. Ref [12]

124. Image
Surface

125. Image of a stereograph
Ref [e]

126. Interpupillary Distance (IPD)

127. Interpupillary Distance
Ref [f]

128. Ref [4]

129. Interpupillary Distance
Ref [f]

130. IPD

131. Gaze

134. Time 1 Time 2

135. Left Eye Right Eye

136. Right Eye

138. Ref [4]

141. Ref [17]

142. Image
Surface

143. Ref [17]

144. Ref [17]

145. Ref [17]

146. Ref [17]

147. Ref [17]

148. Ref [17]

149. Ref [17]

150. Ref [17]

151. Ref [17]

152. Ref [17]

153. Ref [h]

154. Ref [h]

155. Ref [17]

156. Ref [h]

157. First, some terminology...

159. “““Structure from motion (SfM) is a photogrammetric
range imaging technique for estimating three-dimensional
structures from two-dimensional image sequences that may
be coupled with local motion signals. ”””
~Wikipedia

160. Ref [h]

161. Ref [h]

162. And, some more terminology...

163. “““Optical flow or optic flow is the pattern of apparent
motion of objects, surfaces, and edges in a visual scene
caused by the relative motion between an observer and a
scene. ”””
~Wikipedia

165. First section of Figure 12 -- Jump: Virtual Reality Video
Ref [j]

166. Ref [18]

169. Ref [h]

170. Ref [h]

171. Ref [h]

173. Ref [h]

174. Ref [j]

176. Ref [j]

177. Ref [4]

178. Ref [h]

179. Ref [4]

181. Google Cloud
Storage
(GCS)
Capture Stitch
Serve
Browse /
Download
Lead Explore

182. Zeroconf Networking DNS-SD Interpupillary Distance (IPD)
AppleTalk Panorama Field of View (FOV)
NBP Photosphere Optical Flow
mDNS Omnidirectional Stereo Photo Stitching

183. Sources
1. Screenshot from https://www.google.com/search?q=goog
2. Screenshot from https://www.google.com/intl/es419/insidesearch/features/search/knowledge.html
3. See page for author [Public domain], via Wikimedia Commons -- https://commons.wikimedia.org/wiki/File%3AQuicktime-Logo.svg
4. ODS image of Duke Lemur Center captured for Google Expeditions by Google Inc.
5. By Henry Han (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3APershing_middle_school.jpg
6. By User:Gisling (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AAngkor_Wat_from_north_pond.JPG
7. By NASA/Crew of STS-132 [Public domain], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AInternational_Space_Station_after_undocking_of_STS-132.jpg
8. By NASA/Neil Armstrong [Public domain], via Wikimedia Commons -- https://commons.wikimedia.org/wiki/File%3AALSEP_AS11-40-5951.jpg
9. By JonRidinger (Own work) [CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AKent_Roosevelt_science.jpg
10. By U.S. Navy photo by Mike Miller [Public domain], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AUS_Navy_101110-N-0437M-397_Mooseheart_High_School_science_teacher_Curt_Schlinkman_hands_out_homework_to_his_st
udents_who_are_also_Navy_Junior_ROTC.jpg
11. https://en.wikipedia.org/wiki/Routing#Delivery_schemes
12. By Sinchen.Lin [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AASUS_ZenFone_2_CHT_face_20150307.jpg
13. By Evan-Amos (Own work) [Public domain], via Wikimedia Commons -- https://commons.wikimedia.org/wiki/File%3AGoogle-Cardboard.jpg
14. By Rob Janoff [Public domain], via Wikimedia Commons -- https://commons.wikimedia.org/wiki/File%3AApple_Computer_Logo_rainbow.svg
15. By Ben Boldt at English Wikipedia [Public domain], via Wikimedia Commons -- https://commons.wikimedia.org/wiki/File%3APower_Macintosh_5500.png
16. Eadweard Muybridge [Public domain], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3APanorama_of_San_Francisco_by_Eadweard_Muybridge%2C_1878.jpg
17. S. Chen, Quicktime VR An Image-Based Approach to Virtual Environment Navigation, Proc. SIGGRAPH '95, pp. 29-38, Aug. 1995.
18. S. Peleg, M. Ben-Ezra, and Y. Pritch. Omnistereo: Panoramic stereo imaging. PAMI, 23(3), 2001.
19. https://www.pexels.com/photo/binding-black-close-up-dark-436779/

184. Sources
a. By Bill Ebbesen (Transferred from en.wikipedia) [CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3APhotographic_lenses_front_view.jpg
b. By Lionel Brits (Hand drawn in Inkscape by me) [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia
Commons -- https://commons.wikimedia.org/wiki/File%3AEulerangles.svg
c. By Strebe (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AMiller_projection_SW.jpg
d. By Shesmax (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AJesuitenkirche_Franz_Xaver%2C_Luzern._The_nave_(red-cyan_anaglyph)_-_001.jpg
e. http://collections.si.edu/search/results.htm?q=record_ID:chndm_1955-78-118
f. By Tony Hisgett from Birmingham, UK (Owl Eyes 2 Uploaded by Magnus Manske) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3AOwl_Eyes_2_(4572252391).jpg
g. By Bilby (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons --
https://commons.wikimedia.org/wiki/File%3ABaldafix_folding_camera.jpg
h. RICHARDT, C., PRITCH, Y., ZIMMER, H., AND SORKINEHORNUNG, A. 2013. Megastereo: Constructing high-resolution stereo panoramas. CVPR.
i. http://vr.google.com/jump
j. R. Anderson, D. Gallup, J. T. Barron, J. Kontkanen, N. Snavely, C. Hernandez, S. Agarwal, and S. M. Seitz, “Jump: Virtual reality video,” SIGGRAPH Asia, 2016.

185. Real World Virtual Reality

186. Real World in Virtual Reality

187. Questions?

188. @AlexKesling
Alex.Kesling@gmail.com
http://linkedin.com/in/alexkesling

189. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
google-expeditions-virtual-reality