1. Institute for Information Technology
IRET - Multimodal interactive simulator
Institutepublic safety and defence training
for for Information Technology
Dr. Cosmin Munteanu
Cosmin.Munteanu@nrc-cnrc.gc.ca
Heidelberg Innovation Forum – 12 April 2011

2. Old-school training
● Training for urban combat operations requires soldiers to learn
procedures for dealing with numerous real-life situations
● This involves the set up and use of many “props”
● Thousands of plastic or paint bullets
● CO2 area weapons (e.g. flash bangs)
● Paper targets
● Physical rooms / buildings
● Doors breached with explosives
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 2

3. Cost, efficiency, safety
● High cost
● Current training resources are not always reusable (e.g. CO2
cartridges, plastic bullets, broken doors)
● Inefficient
● Layouts are not easily reconfigurable (wooden rooms requiring
carpentry work to reconfigure or even concrete buildings)
● Lack of realism
● Paper targets are static
● Safety needs
● Certain operations require extreme caution (e.g. breaching a door
with a C4 explosive charge)
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 3

4. Virtual training with the
IRET-MINT environment
● The IRET-MINT training platform addresses issues of cost,
efficiency, realism, and safety faced by current urban operations
training
● It is a mixed-reality, immersive
environment and gaming platform
supporting customizable training
scenarios for defence, public
safety, and law enforcement, in a
realistic and controlled setting
● Developed by researchers in HCI, ASR, NLP, SysEng at the
National Research Council in collaboration with the Department
of National Defence / Combat Training School
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 4

5. The IRET-MINT system
● Flexible training platform for easy reconfiguration of the
immersive environment to support different training scenarios
● Universal communication protocol facilitating integration of
external components and network-enabled remote training
● Multimodal interaction
(speech, gestures, etc.)
● Electronic simulated
area weapons
● Integration of other weapons
(e.g. NRC's own LaserTrak)
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 5

6. Multimodal interaction in IRET-MINT
● Plug-in interface for speech recognition and language
processing for natural interaction with virtual characters –
flexible and robust processing of speech under stress
● Domain-specific electronic devices simulating lethal and non-
lethal weapons that interact with the IRET platform
● Cost-effective and safe
● Increased efficiency and
effectiveness of training
● Example: flashbang, explosive
door breacher, etc.
● Tablet-based control interface for
instructors allowing real-time manipulation of training scenarios
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 6

7. Solutions for current and future needs
● We have developed IRET-MINT:
● Immersive, interactive environment for urban ops training
● Platform enabling natural interactions between users and
intelligent, mixed-reality environments
● Flexible, cost-effective, safe solution
● Our core technologies provide R&D and licensing opportunities
in a wide range of industries (public safety, law enforcement,
defence training)
● We continually seek industry partners to:
● Develop the next iterations of the system
● Broaden the scope of this technology
● Reach relevant markets world-wide
Cosmin Munteanu
Multimodal Interaction for Mixed-Reality Training: the IRET-MINT Platform 7

8. Thank you!
● Acknowledgments:
● NRC IIT Research Team: Irina Kondratova, Cosmin Munteanu,
Haozheng Li, Muhammad Al-Digeil, Brian McKinnon, Ilia Goldfarb,
Rene Richard, Cedric Martin, Nathan Langton, Joanna Lumsden,
Ryan Nielson, Yasaman Sanaee
● Department of National Defence (DND)
● CFB Gagetown – Combat Training Center Tactical School
Questions, discussions?