Top 10 scenario-based questions for hiring a Team Leader
EMIP15 Workshop Presentation
1. Studying Sustained Attention & Cognitive
States in Remote Technical Interviews
Denae Ford, Titus Barik, Chris Parnin
North Carolina State University
5. Nonverbal Cues & Think-Aloud
Hollandsworth (1979) found that interviewers place high importance on
nonverbal cues such composure and eye contact
Can collect these nonverbal cues that occur during the interview process
through eye tracking
Confirm them through the “eye-mind hypothesis”
6. Monitoring Sustained Attention
"the ability to direct and focus cognitive activity on specific stimuli”
What if we used the concepts from sustained attention to enhance
programming under high stress situations?
7. Pupil Dilation
Large pupil dilation implies high
cognitive load
Should not deter attention of candidate
during this peak processing time
Can break concentration & make it difficult to
resume task
Source: https://upload.wikimedia.org/wikipedia/commons/a/a3/Eye_dilate.gif
8. Saccades
Connections between shifts of attention & saccades
Saccades occur involuntary during information processing
If saccades begin to stray from information processed indicates a stray in
sustained attention
Source: http://49.media.tumblr.com/tumblr_md36ruuNXk1qfg7o3o1_400.gif
9. Blinks
Help determine a candidate's’
interest in a task
Using size and speed of blink we can
determine if the attention of the
candidate is on the eliciting
stimulus
Source: http://45.media.tumblr.com/91cfc58c55c969b876ab0238ddcda507/tumblr_mss1ejaDpX1r3maj7o1_500.gif
11. Proposed Study
C1: No Interventions
E2: Blackout Only
E3: Focus Light Only
E4: Both Blackout and Focus Light
12. Challenges
Determining what valuable information can we extract from using eye
tracking in technical interviews
Seems like this would be a great example to see someone program under a stressful
situation
13. Conclusion
Only one step to understanding technical interviews
Goal: eventually establish a golden set of adaptable technical interviews for
different programming styles
Source: http://blog.rocketclub.co/wp-content/uploads/2015/03/bad_interview_questions.jpgSource: https://coderpad.io/
16. Research Questions
RQ1: Is the candidate able to sustain their attention to the programming
issue less when the interviewer has increased view of their actions?
RQ2: How does visibility of the interviewer affect the candidates’
performance?
RQ3: How is the performance of novices and experts reflected in their
cognitive states during interviews?
17. How Realistic is This?
There are smart phones that take advantage of this technology
Starting with Samsung Galaxy S4 - Eye Scroll Feature, SmartStay
Matter of time before it’s widespread such as laptops like Tobii
Reflecting on this it makes sense why many of these companies would prefer to do phone interviews. It saves money!
“However, they're missing that visibility affordance you get when you're in the same room as a person.
You're missing out on nonverbal cues that allow the interview to go seamlessly”
http://2.bp.blogspot.com/-nQc-Dj517_o/UTns6klEtTI/AAAAAAAAAEw/F9nK5uSVUBQ/s1600/people_talking_on_phone.jpg
“The visibility gives interviewers unconscious notice as to when the candidate may be nervous, working through the programming task given, and even the appropriate time to solicit for the candidate’’ thought process.”
Source: http://www.clker.com/cliparts/5/b/9/8/1194984513646717809chat_icon_01.svg.hi.png
Eye mind hypothesis: people are paying attention to what they’re looking at.
“Programming is a sustained attention task. Not only are you typing the code but you are comprehending the problem and devising an approach to solve it.”
Sustained attention is "the ability to direct and focus cognitive activity on specific stimuli." In order to complete any cognitively planned activity, any sequenced action, or any thought one must use sustained attention. An example is the act of reading a newspaper article. One must be able to focus on the activity of reading long enough to complete the task. Problems occur when a distraction arises. A distraction can interrupt and consequently interfere in sustained attention.
DeGangi and Porges (1990) indicate there are 3 stages to sustained attention which include: attention getting, attention holding, and attention releasing.
Sustained attention is important to psychologists because it is "a basic requirement for information processing." Therefore, sustained attention is important for cognitive development. When a person has difficulty sustaining attention, they often present with an accompanying inability to adapt to environmental demands or modify behaviour (including inhibition of inappropriate behaviour).
When pupils are dilated you can know that the applicant is processing info. Not a good time to interrupt them
Shifty eyes mean anxious and you can study the rate of saccades to monitor info processing as well.
Low Blink rates can indicate high focus and also Hugh blink rate can indicate anxiety
Blinks can help us determine
“All these pupillary responses and eye movements can help determine where the attention of the programmer is and the appropriate time to deter it”
Visibility can be answered multiple ways (with the multiple version of interviews (phone, in person, can see code)
or visibility as in has control over visibility or not with blackouts