"Serious Communication for Serious Games" For decades, military and commercial aviation have been using flight simulators to help teach pilots to fly. Over the years, it has been shown that augmenting real-world training with virtual training results in cheaper, safer, and more effective training. These and other training devices have spread and evolved and now can be found throughout the military being used to train a wide variety of individual skills as well as complicated joint-operation teamwork skills such as convoy operations and Call for Fire. This talk will explore some of the technological challenges faced when building high-fidelity multiplayer training games for a global dynamic training network. Stringent military requirements include linking disparate training devices together such as serious games, full-fidelity flight simulators, and live-fire ranges so that soldiers may train in real-time with other units around the world in the same virtual world.

1. Ross Kukulinski
Serious Play Conference 2013
1

2. 2
Photo by Derek Jensen

3. 3
Photo Courtesy of U.S. Military

4. 4
Serious Games
are Powerful
Tools

5. Serious Games …
 Allow soldiers to experience situations that
are impossible in the real world1
 Provide improved hand-eye coordination,
multi-tasking, and teamwork2
 Are uniquely flexible to support varied
training needs
5
1 Corti, 2006; Squire & Jenkins, 2003
2 Michael & Chen, 2006

6. Fundamentals of Teamwork
The Big Five Core Components of Teamwork1
1. Team Leadership
2. Performance Monitoring
3. Backup Behavior
4. Adaptability
5. Team/Collective Orientation
Hypothesis: Communication key element?
6
1 Salas, Sims, & Burke, 2004

7. Communication and
Performance
 America‟s Army experiments
 Researchers measured team communication
○ Communication network level
○ Number of report-ins
○ Number of normal communications
 Teams with regular organized reports had:
 Higher performance
 Higher estimated situational awareness
7
Schneider & Carley, 2005

8. DARWARS Ambush!: Authoring
Lessons Learned in a Training
Game1
 Communication skills are critical for success
 Communications capabilities differ widely
across varying military units
 Training system should be similar to real-
world communication system
8
1 Diller, Orberts, Blankenship, Nielsen, 2004

9. „Good‟ Serious Games
Six Ingredients to a good game1
1. Mechanics
2. Rules
3. Immersive Graphics
4. Interactivity
5. Challenge
6. Risks
7. What about communication?
9
1 Derryberry, 2007

10. 10
Team Building
Examples

12. U.S. Army Grafenwoehr,
Germany
12
Photo Courtesy of U.S. Military
First responder training with Live & Virtual Training

13. Live-Virtual-Constructive
13

14. 14

15. Incident-Management Training
15

16. A Training Transfer Study of
Serious Games
“Our work in this project demonstrated
consistently through all five experiments
that communications is fundamental to
the training experience and one of the
most important aspects of the exercise.”
Major Ben Brown
MOVES Institute
Naval Postgraduate School
16
Brown, 2010

17. On a personal note:
 My biggest failures typically come down
to one of two things:
 I didn‟t communicate
 Or I didn‟t communicate effectively
17

18. So what?
 If we‟re building games to train or teach
 and teamwork is a determining element of
success or failures
 Then our games need to accurately
reflect real-world communication
18

19. 19
Real-world
Communication

20. 20
What types of communication
do you use in your workplace?

21. My short list:
 Email
 Instant Messaging
 Telephone / Cell Phone
 VoIP / Video Calling
 Face-to-face
 Twitter
21

22. Acceptable Fidelity
• What is the training goal?
• What is the real-world communication?
• “One can debate the level of fidelity
needed for useful training, but fidelity
must certainly be high when it relates
to the specific task being trained”
22
1Brown, 2010

23. Types of fidelity
 Communication simulation
 Communication user interface
23

24. 24
Communication
Simulation

25. And now for a military
example…
25

26. Basic Intercom
26

27. Intercom and Individual Radios
27

28. Geolocated Individual Radios
28

29. Geolocated Vehicle & Individual
Radios
29

30. Highest Fidelity
30

31. Team building examples:
 Compound & IMTS
 „Intercom‟ only for all players
 Grafenwoehr First-Responders
 Simulated radios & Earshot
31

32. 32
Communication
User Interface

33. “Quite simply, communications should be
as seamless as all other aspects of [the
serious game]. Communications should
be internal to [the game] with seams
between vendor production transparent
to the user.”1
Major Benjamin Brown,
MOVES Institute,
Naval Postgraduate School
33
1Brown, 2010

34. Depends on fidelity
 Players shout over their monitors
 In-game text-chat
 Simple press-to-talk key for voice
 Or communication „items‟ are playable
objects
 Intelligent agents?
34

35. User Interface
 Heads up display
 Simple and intuitive
 Flashing icon over avatar heads
 Avatars‟ mouths move
 Not realistic – does it break flow?
 In-game objects
 e.g. walk-up to a virtual computer and
interact with it
 Higher realism – but does it impede
training?
35

36. Regardless of design
decision
 Quality of the audio is paramount
 Dropped or garbled audio is not acceptable
 Scalability can really be an issue
 Latency also matters
36

37. Audio Latency
• End-to-end Latency
– Time for audio to travel from one user to another
• Effected by many factors
– Network link
– Packetization delay
– Operating system delay
– Hardware device delay
• Maximum 150ms one-way latency1
• Latency <100 ideal
37
1ITU-T G.114

38. 38
After Action
Review

39. After Action Review
• “Both simulation groups commented
extensively on the AAR tool. Both
groups believed the AAR tool was
critical in providing a big picture view of
what happened during the exercise.” 1
39
1Brown, 2010

40. After Action Review
• Communication playback synced with
visuals
• Seek, Pause, FF, RW, Bookmarks
• Export audio/visual for later analysis and
study
• BIG data?
40

41. 41
Final Thoughts

42. A Common Myth
 High fidelity means hard to use
 (and expensive?)
However:
 Does require insight into operational
environment
42

43. Summary
 Communication is critical for teamwork
 Serious games require high-fidelity and
high-quality communication for effective
team-based training
 Repetition is important, but so is
analysis
43

44. Resources
 Brown, B., (2010) A Training Transfer Study of Simulation Games
 Carpenter, R., White, C., (2005) Commercial Computer Games in the Australian
Department of Defense
 Corti, K. (2006) Games-based Learning; a serious business application.
 Derryberry, A. (2007) Serious Games: online games for learning
 Diller, D., Roberts, B., Blankenship, S., Nielsen, D. (2004) DARWARS Ambush!
Authoring Lessons Learned in a Training Game
 Hussain, T., etal (2010) Development of game-based training systems: Lessons
learned in an inter-disciplinary field in the making
 Hussain T. & Ferguson, W. (2005) Efficient Development of Large-Scale Military
Training Environments using a Multi-Player Game
 McGowan, C., Pecheux, B. (2007) Serious Games that Improve Performance
 Michael, D., & Chen, S. (2006) Serious games: Games that educate, train and
inform
 Sims E., Salas E., Burke S. (2004) Is There a „Big Five‟ in Teamwork
 Snider, M., Carley K., Moon, I. (2005) Detailed Comparison of America‟s Army
and Unit of Action Experiments
 Squire, K. & Jenkins, H. (2003) Harnessing the power of games in education
44

45. Thank You!
Ross Kukulinski
@rosskukulinski
ross.kukulinski@asti-usa.com
45