Chris Grayson and Heidi Hysell gave this duo presentation at SXSW Interactive, 2012. Full audio and information can also be viewed here: http://schedule.sxsw.com/2012/events/event_IAP13307

1. NOTES

2. NOTES: (Video & Audio)
Douglas Engelbart speaking as attendees arrive.

3. NOTES:
Welcome.

4. NOTES:
Introduction of Chris Grayson.

5. NOTES:
Introduction of Heidi Hysell.

6. NOTES:
What is Augmented Reality?
AR is not a technology unto itself, but a collection of technologies brought together to
achieve a given result, so it is not a simple question to answer.

7. NOTES: (shown: Bruce Sterling, Vernor Vinge, Amber Case)
Bruce asked the same question of attendees at the Augmented Reality Event keynote
speakers’ dinner in May of last year. It would be the subject of his opening keynote. At
the time, my self-serving special effects house answer was, “Special effects, rendered
in real time, over real life.” A broader definition might be, “Interjection of virtual content into one’s
sensory perception, within the context of the real world.”

8. NOTES:
For the purposes of our presentation, we are going to make some assumptions.
Augmented Reality defined as some combination of these four technologies, when used
together.

9. NOTES:
Also — What this talk is, and is not.
It is not a claim that ARPA invented Augmented Reality half a century ago and we’re
just catching up with them. It is, however, an argument that many of the core concepts
behind what we know of as AR were part-and-parcel of the research being funded by ARPA,
and furthermore, that this work was being done without our preconceived notions about what a
personal computer is. Human augmentation was, in fact, at the core of their research.

10. NOTES:
Our story begins in 1957.

11. NOTES:
Elvis was the top recording artist in the country.

12. NOTES:
Cadillacs still had great fins.

13. NOTES:
Ian Fleming’s James Bond novel, From Russia With Love, was published.

14. NOTES:
And the Soviets launched Sputnik.

15. NOTES:
The US responded by creating ARPA, the Advanced Research Projects Agency, with
the mandate to fund rocket and missile research, and to put satellites into space.

16. NOTES:
Within months, the US also created the civilian agency, NASA.

17. NOTES:
This was all during the Eisenhower administration. Eisenhower was an advocate of
NASA.
From the start, ARPA faced a crisis of purpose: Its mandate was taken from it — rocket research
was transferred to NASA, missile research was assumed by the various branches of the military,
and ARPA was adrift. The need for its very existence questioned.

18. NOTES:
In 1962 ARPA’s third Director, Jack Ruina, was busy managing the design of an
incoming-missile early warning system. He was tasked by the Pentagon to create two
new departments ...

19. NOTES:
Bother new departments would fund research into computer / human interaction.
One was a Behavioral Sciences department, and the other was for Command and
Control.
Jack believe he had the perfect candidate to lead both, and they would shortly be merged into
what became the IPTO: the Information Processing and Techniques Office.

20. NOTES:
Tell anecdote: “JCR Licklider, but you can call me ‘Lick’.”
Tell anecdote: Much to colleagues dismay, Licklider resigned from MIT the year he
made tenure. After joining Bolt, Beranek & Newman, he would shift again to the public sector, as
the founding director of the IPTO.

21. NOTES:
There was another candidate: Licklider’s friend Fred Frick, of MIT’s Lincoln Laboratory.

22. NOTES:
Tell anecdote: Coin Toss.

23. NOTES: (Audio)
Not that kind of “Psycho” ... or that kind of “Acoustics.”

24. NOTES:
Licklider’s background was in Psycho-Acoustics.
Not an electrical engineer, or engineer of any kind. An HR department would have never
considered him for this position.

25. NOTES:
His qualifications came from a paper he wrote in 1960, Man Computer Symbiosis.

26. NOTES:
According to Licklider — henceforth, man and computer would co-evolve ... this, at a
time when computers were rare, huge and expensive; punchcards were the common
interface, and even by those who used them, were generally looked upon as little more
than glorified calculators.

27. NOTES:
Computers
Text here. viewed as glorified calculators.

28. NOTES:
1930s — A “Computer” was a title given to women who did computations by calculator.

29. NOTES:
1936 — Alan Turing creates the Turing machine, a conceptual machine for binary
computation and algorithms.

30. NOTES:
1946 — transition from mechanical analog to electrical analog. Eniac has a patch cable
interface. In its lifespan, Eniac alone did more calculations than all of mankind up until
that point in human history.

31. NOTES: (Audio)
The Daleks. Just kidding ... they didn’t arrive until 1963.

32. NOTES:
1949 — The Manchester Mark 1 had the first stored computer program heralding the
birth of the software industry.

33. NOTES:
Grace Hopper created the first compiler.
Tell anecdote: Coined the term “bug” after finding a moth inside the computer.

34. NOTES:
Fortran was the first successful high level programming language.

35. NOTES:
IBM used punch cards as their standard input/output device.

36. NOTES:
Due to ubiquity of punchcards, magnetic storage had a very slow adoption rate.

37. NOTES:
The Univac was the first commercially available computer with magnetic storage and
direct keyboard access.
Tell anecdote: Though innovative, not a commercial success, due to companies’ large
investments in existing IBM standards, including punchcards.

38. NOTES:
Direct Keyboard Input.

39. NOTES:
1958 —Light Gun interface, shown here used on SAGE (Semi Automatic Ground
Environment) aircraft tracking system.

40. NOTES:
One point worth noting about Licklider’s Man Computer Symbiosis is that he made clear
this was not a “tool” relationship. The computer was not the newest tool along the lines
of the wheel, or the locomotive, this was true symbiosis.

41. NOTES:
He anticipated a time when our interface with computers would be through talking to
them. It cannot be overstated, that this was in the time of punch-cards.

42. NOTES:
Possibly the most important thing Licklider brought to ARPA was agency culture.
He was nicknamed, Johnny Appleseed.
Tell anecdote: Declined purchase of or building of agency laboratories, choosing instead to
fund research done at different universities and other institutions. It did not have to be this way.
Licklider established that, and it has been maintained, now via DARPA, to this day.

43. NOTES:
Tell anecdote: Licklider’s Intergalactic Computer Network.

44. NOTES:
ARPA financed Artificial Intelligence research done by Marvin Minsky, inventor of the
world’s first artificial neural network.

45. NOTES:
Due to its discretion in dispersing money, ARPA had a huge influence over what kind
of research was done, and what work was not, and the director of the IPTO could
essentially steer research across the field.

46. NOTES:
Minsky was also the mentor to a young Ivan Sutherland.

47. NOTES:
For his PhD thesis, Sutherland created Sketchpad.

48. NOTES:
Then Sutherland became the 3rd Director of the IPTO. Shown here, he took an idea of
his mentor’s, the head mounted display, and saw it to actualization. It was stereoscopic,
and transparent (viewing virtual content over one’s view of the real world).
Tell anecdote: My phone call with Ivan Sutherland in 2011, attempting to hire him as a speaker.

49. NOTES:
Sutherland’s description of the use of a transparent lens head mounted display from his
paper, The Ultimate Display, gives what is still one of the most common use-cases for
advanced AR eyewear.

50. NOTES:
Industry transitioning from vacuum tubes to transistors: faster, reliable, smaller, cheaper.

51. NOTES:
The invention and adoption of the integrated circuit. Gordon Moore (cofounder of Intel)
and Moore’s Law: The number of transistors that can be placed on an integrated circuit
doubles approximately every two years.

52. NOTES:
Two breakthrough technologies.
Hardware industry core concepts haven’t changed much.

53. NOTES:
Tell anecdote: Software growing pains.
More affordable computers, programming overshadows hardware costs.

54. NOTES:
Creation of higher level, user friendly programming languages.
Writing programs becomes easier.
More complex problems, cost is still huge.

55. NOTES:
Leonard Nimoy and his 1964 Buick Riviera.
Tell anecdote: Getting Star Trek date wrong.

56. NOTES:
Was poached by ARPA from NASA.
Taylor was first interviewed by Licklider, and hired by Sutherland to be his Deputy
Director. He was groomed to take the helm — the succession plan.

57. NOTES:
Tell anecdote: Amazing parallels in Taylor & Licklider’s lives.
Both were sons of ministers. Both were psychologist. Both studied psycho-acoustics.
Got his masters in psycho-acoustics at The University of Texas at Austin.

58. NOTES:
Another thing Licklider and Taylor had in common: They were both already funding
research being done at the Stanford Research Institute.

59. NOTES:
Specifically, they were both funding the research of Douglas Engelbart.

60. NOTES:
At his Augmentation Research Center.
Tell anecdote: The invention of the mouse (with Bill English).

61. NOTES:
Douglas wrote an influential paper, itself influenced by Licklider’s Man Computer
Symbiosis, titled Augmented Human Intellect, a Conceptual Framework.
And as the name says, it is a framework.

62. NOTES:
In it, he makes the case for advancing human-computer interface design, whereby
computers would come to be “Augmentation Systems” to the human mind.

63. NOTES:
All of this work culminated in a presentation that has now come to be know as the
Mother of All Demos, performed by Douglas Engelbart at the Fall Joint Computer
Conference held at the San Francisco Convention Center in December of 1968.
First public demonstration of a Mouse, of hyper-text, of video conferencing, and ...

64. NOTES:
The first public announcement of the ARPAnet, predecessor to the Internet.
Tell anecdote: How Bob Taylor got the financing to build the ARPAnet in one
20 minute conversation with Charlie Herzfeld.

65. NOTES: (Video & Audio)
Douglas Engelbart, in his own words.

66. NOTES:
1. Ivan Sutherland’s head mounted display.
2. JCR Licklider’s Man Computer Symbiosis.
3. Douglas Engelbart’s Augmented Human Intellect & Augmentation Research Center.
4. Marvin Minsky’s artificial intelligence research.
But it’s clearly more than just a checklist.

67. NOTES:
Some post ARPA history.
Bob Taylor became the Founding Director of Xerox Parc Research Center, poaching a
lot of talent from Douglas Engelbart, in the process.
Tell anecdote: Bob Taylor on the myth that Xerox didn’t capitalize on Xerox Parc.
Tell anecdote: Where Steve Jobs first saw the graphical user interface that inspired the Mac.

68. NOTES:
Augmented Reality becomes a pop-culture meme as “Terminator Vision.”

69. NOTES:
Being engineer, Tom Caudel says he coined the term “Augmented Reality” in 1991.
Tell anecdote: Caudel, safety goggles, and the Boeing assembly line.

70. NOTES:
The same year that Caudel claims to have coined the phrase (some even say,
“invented”) Augmented Reality, the top grossing action film was, Terminator 2 — Where
might one have come up with such an idea as virtual content over one’s view of reality?

71. NOTES:
In the mid-90s Paul Milgram and Haruo Takemura publish a paper that introduces what
has come to be known as Milgram’s Continuum.

72. NOTES:
By the late 90s the first functional mobile Augmented Reality system was developed for
the PhD project of Blair MacIntyre and his professor, Steve Feiner. Blair is now Director
of the Augmented Environments Lab at Georgia Tech, and Steve continues to teach
Computer Science at Columbia.

73. NOTES:
Recent Developments in AR eyewear: Google

74. NOTES:
Recent Developments in AR eyewear: Apple patents

75. NOTES:
A Vision for what’s to come.
Before dedicated AR eyewear ever have potential to go mainstream, I expect to see
Bluetooth mobile accessories ... for your eye.

76. NOTES:
Was impressed with the form factor of the Docomo/Olympus collaboration from 2011.
Whether by docomo/olympus, or another brand or team, this is the form factor I expect
the first successful eyewear to take. I also speculate we will see something in market in the next
12 to 18 months.

77. NOTES:
Another view.

78. NOTES:
And another.

79. NOTES:
Some noteworthy companies operating in the AR space today.

80. NOTES:
You can learn more about the state of AR at Augmented Reality Event, held this year
on May 8-9, 2012, at the Santa Clara Convention Center, in Santa Clara, CA.

81. THANK YOU