1. Emotional Impacts of Digital Media Robert Atkinson Maria-Elena Chavez-Echeagaray Robert M. Christopherson David Gibson Javier Gonzalez-Sanchez This research was supported by Office of Naval Research under Grant N00014-10-1-0143 awarded to Dr. Robert Atkinson

2. Research Questions At what moments does a digital media user experience identifiable physical, emotional and cognitive attributes? Are those facilitating or impeding moment-by-moment learning and performance?

3. Research Questions What patterns are found within & between sensors? How do these patterns relate to baseline and experimental activities?

4. Sensors Wireless EEG Facial muscles, emotional clusters, raw EEG Wireless Galvanic Skin Conductance Arousal level Eye Tracker Gaze-point, duration, mouse-clicks Haptics Button presses, head tilt

5. Anatomy of the System

6. Anatomy of theSystem Wireless Emotiv® EPOC Headset(report data with intervals of 125 ms). The output includes 14 sensors or channels (7 on each brain hemisphere: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4) and two values of the acceleration of the head when leaning (gyrox and gyroy). Also it reports Engagement, Boredom, Excitement , Frustration, Meditation. Javier Gonzalez-Sanchez | Maria-Elena Chavez-Echeagaray

7. Anatomy of theSystem Tobii®Eye Tracker report data with intervals of 100 ms. provides data concerning attention direction and time of focus during individual use of a computer. Javier Gonzalez-Sanchez | Maria-Elena Chavez-Echeagaray

8. Anatomy of theSystem MindReader Software from MIT Media Lab. This is about inferring a person mental state from non-verbal cues. Visual system infers mental states from head gestures and facial expressions in a video stream in real-time at data intervals of 100 ms approximately. This system infer six emotions: agreeing, concentrating, disagreeing, interested, thinking and unsure. Javier Gonzalez-Sanchez | Maria-Elena Chavez-Echeagaray

9. Anatomy of theSystem Skin electrical conductance sensordesigned by MIT Media Lab for Arousal Sensing. Reports conductance data in intervals of 500 ms This sensor measures the electrical conductance of the skin, which varies with its moisture level that depends on the sweat glands, which are controlled by the sympathetic, and parasympathetic nervous systems. Javier Gonzalez-Sanchez | Maria-Elena Chavez-Echeagaray

10. Method 20 users play Guitar Hero sessions Easy and hard songs 10 minutes of recording per session Over 76000 records per user per session

11. Video

12. Data

13. Thinking States Rise in uncertainty and interest During thinking Agreement & concentration drop

14. Analysis Inductive structural equation modeling

15. Analysis Network analysis Adjacency tables Centrality Digraphs

16. Analysis Digraphs illustrate structural relationships in the causative factors during a time slice.

17. Findings Near real-time displays are possible that relate to hidden psychological variables Time slices of performance can be analyzed to give statistically accurate operational models of brain states during specific tasks

18. Findings Learner analytics can use these analyses to Personalize digital media experiences Control devices (e.g. cars, computers, lighting) Study emotion and learning Research seductive details in hypertexts Improve affective & metacognitive tutors