Background and initial experiments performing subvocalization studies of programmers with EMG recordings.

1. SubvocalizationToward Hearing the Inner Thoughts of Developers Chris Parnin Georgia Tech

2. Overview + = Subvocal speech EMG (electromyography) Cognition?

3. Problem

4. Questions Does a tool or language reduce cognitive effort? Can we identify moments of learning uncertainty? Can we distinguish cognitive processes? Can we find news ways to interact?

5. Some Attempts of Measuring Cognition

6. fMRI 12-30 secondstasks While crammed in the middle of a jet engine

7. EEG

8. History of Reading? ”His heart sought out the meaning, but his voice was silent and his tongue was still” 300s AD

10. Kymograph

11. Inner Speech (Brain)

13. Speech Plans and Acts

14. fMRI processing inner speech..

15. BRAIN

16. Measure with EMG

17. Sokolov’s EMG Recordings (1970s)

18. Reasons why People Subvocalize Attention Focus, maintain Boost Multiple modalities Activation comprehension, memory

19. Research Questions When do developers subvocalize? Can subvocalization be used to measure difficulty of programming task? Distinguish visual or verbal cognitive processing?

20. Tasks Make block fall all the way down with space key. Make monster reappear if missed shot.

21. An instrumented session…

22. Moments of interest and difficulty Formulating Problem Found useful code Editing Formulating solution Found useful code Evaluating

23. Difference in tasks at frequency bands microvolts Frequency bands

24. One more thing…

25. Silent Speech Interfaces

26. State of the Art ? “Things Programmers Think When Coding?” Jorgensen 6 words (8% error) Schultz 101 words (10% error) 2003 2010 ICPC 2012

28. Questions? Detect Hmm?