This document discusses using machine learning to analyze individual and interpersonal behavior for clinical diagnosis and screening. It focuses on analyzing non-verbal behaviors like interpersonal synchronization that have been shown to be impaired in conditions like autism spectrum disorder. The document proposes that machine learning could provide an objective, automated tool for diagnosing conditions more quickly by analyzing video recordings of social interactions. This may help address bottlenecks in healthcare systems and allow earlier access to treatment.

1. ML for Clinical Diagnosis
Visual Analysis of
Individual and Interpersonal Behavior
Mircea Stoica
Heldenkombinat Technologies GmbH

2. Jorge Davila-ChaconOlaf Erichsen

3. Autism Spectrum
Disorder

4. HELDENKOMBINAT
MACHINE LEARNING
4
ASD prevalence
Visualization by The Autism Community in Action https://tacanow.org/autism-statistics/
Based on data from the ADDM report by the Centers for Disease Control and Prevention, US (CDC)

5. HELDENKOMBINAT
MACHINE LEARNING
5
Screening and diagnosis
• Early screening is crucial
• Urgent need for a faster and objective automatized diagnostic tool
• Alleviate bottlenecks in the healthcare system
• Earlier access to benefits, specific interventions and other healthcare
programs
• A relief in the demand of health care resources associated with mental
health comorbidities in ASD
Fein, D., Barton, M., Eigsti, I. M., Kelley, E., Naigles, L., Schultz, R. T., … Tyson, K. (2013). Optimal outcome in individuals with a history of
autism. Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(2), 195–205. https://doi.org/10.1111/jcpp.12037

6. HELDENKOMBINAT
MACHINE LEARNING
6
Screening and diagnosis
• Collaboration with King’s College London:
Dr Alexandra L. Georgescu
• ML for the screening and diagnostic of Autism
• 1.1% in the UK may be on the autism spectrum
Georgescu, A.L., Koehler, J.C., Weiske, J., Vogeley, K., Koutsouleris, N, Falter-Wagner, C. (2019). Machine learning approaches
to study social interaction difficulties in Autism Spectrum Disorder. Frontiers in Robotics and AI. Under review.
Kuschefski, M., Falter‐Wagner, C. M., Bente, G., Vogeley, K., & Georgescu, A. L. (2019). Inferring power and dominance from
dyadic nonverbal interactions in autism spectrum disorder. Autism Research, 12(3), 505-516.

7. HELDENKOMBINAT
MACHINE LEARNING
7
Social interaction

8. HELDENKOMBINAT
MACHINE LEARNING
8
Social interaction
• A shift in research
• May provide a relevant marker
• Interpersonal synchronization
Sebanz, N., Bekkering, H., & Knoblich, G. (2006). Joint action: bodies and minds moving together. Trends in Cognitive Sciences, 10(2),
70–76. https://doi.org/10.1016/j.tics.2005.12.009
Leong, V., & Schilbach, L. (2019). The promise of two-person neuroscience for developmental psychiatry: using interaction-based
sociometrics to identify disorders of social interaction. The British Journal of Psychiatry, (July), 1–3.
https://doi.org/10.1192/bjp.2019.73

9. HELDENKOMBINAT
MACHINE LEARNING
9
Interpersonal synchronization

10. HELDENKOMBINAT
MACHINE LEARNING
10
Interpersonal synchronization
Isenhower, R. W., Marsh, K. L., Richardson, M. J., Helt, M., Schmidt, R. C., & Fein, D. (2012). Rhythmic bimanual coordination is
impaired in young children with autism spectrum disorder. Research in Autism Spectrum Disorders, 6(1), 25–31.
https://doi.org/10.1016/j.rasd.2011.08.005
Fitzpatrick, P., Frazier, J. A., Cochran, D. M., Mitchell, T., Coleman, C., & Schmidt, R. C. (2016). Impairments of social motor synchrony
evident in autism spectrum disorder. Frontiers in Psychology, 7(AUG), 1–13. https://doi.org/10.3389/fpsyg.2016.01323
Kaur, M., M. Srinivasan, S., & N. Bhat, A. (2018). Comparing motor performance, praxis, coordination, and interpersonal synchrony
between children with and without Autism Spectrum Disorder (ASD). Research in Developmental Disabilities, 72(February
2018), 79–95. https://doi.org/10.1016/j.ridd.2017.10.025
Fitzpatrick, P., Romero, V., Amaral, J. L., Duncan, A., Barnard, H., Richardson, M. J., & Schmidt, R. C. (2017). Social Motor
Synchronization: Insights for Understanding Social Behavior in Autism. Journal of Autism and Developmental Disorders, 47(7),
2092–2107. https://doi.org/10.1007/s10803-017-3124-2
Fitzpatrick, P., Romero, V., Amaral, J. L., Duncan, A., Barnard, H., Richardson, M. J., & Schmidt, R. C. (2017). Evaluating the
importance of social motor synchronization and motor skill for understanding autism. Autism Research, 10(10), 1687–1699.
https://doi.org/10.1002/aur.1808

11. HELDENKOMBINAT
MACHINE LEARNING
11
Interpersonal synchronization

12. HELDENKOMBINAT
MACHINE LEARNING
12
Interpersonal synchronization
Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2018). 3D human pose estimation in video with temporal convolutions and semi-
supervised training. Retrieved from http://arxiv.org/abs/1811.11742

13. HELDENKOMBINAT
MACHINE LEARNING
13
Interpersonal synchronization

14. Individual and Interpersonal
Behaviour

15. HELDENKOMBINAT
MACHINE LEARNING
15
Screening and diagnosis
• ML for the analysis of individual and interpersonal non-verbal
behaviour in clinical patients
• Interpersonal behavior is associated with other conditions:
• Schizophrenia (Varlet et al., 2012, Kupper et al., 2015)
• Social anxiety disorder (Varlet et al., 2014)
• Borderline personality disorder (Gratier)
Leong, V., & Schilbach, L. (2019). The promise of two-person neuroscience for developmental psychiatry: using interaction-
based sociometrics to identify disorders of social interaction. The British Journal of Psychiatry, (July), 1–3.
https://doi.org/10.1192/bjp.2019.73

16. HELDENKOMBINAT
MACHINE LEARNING
16
Screening and diagnosis
• Varlet M, Marin L, Raffard S, Schmidt RC, Capdevielle D, Boulenger J-P, et al. Impairments of Social Motor Coordination in
Schizophrenia. PLOS ONE. 2012 Jan 17;7(1):e29772.
• Kupper Z, Ramseyer F, Hoffmann H, Tschacher W. Nonverbal Synchrony in Social Interactions of Patients with Schizophrenia
Indicates Socio-Communicative Deficits. PLOS ONE. 2015 Dec 30;10(12):e0145882.
• Varlet M, Marin L, Capdevielle D, Del-Monte J, Schmidt R, Salesse R, et al. Difficulty leading interpersonal coordination:
towards an embodied signature of social anxiety disorder. Front Behav Neurosci [Internet]. 2014 [cited 2019 Jul 5];8.
• Gratier M, Apter-Danon G. The musicality of belonging: repetition and variation in mother-infant vocal interaction, in
communicative musicality: narratives of expressive gesture and being human. In: Communicative Musicality: Exploring the
Basis of Human Companionship. Oxford: Oxford University Press; 2008. p. 301–27.
• Romero V, Fitzpatrick P, Roulier S, Duncan A, Richardson MJ, Schmidt RC. Evidence of embodied social competence during
conversation in high functioning children with autism spectrum disorder. PLOS ONE. 2018 Mar 5;13(3):e0193906.
• Noel J, Niear MAD, Lazzara NS, Wallace MT. Uncoupling Between Multisensory Temporal Function and Nonverbal Turn-
Taking in Autism Spectrum Disorder. IEEE Trans Cogn Dev Syst. 2018 Dec;10(4):973–82.

17. Take away

18. HELDENKOMBINAT
MACHINE LEARNING
18
Take away
• ML can save years (decades?) of work to domain experts:
the algorithms discover the patterns for them.
• Deployment in far less-constrained environments.
• Same methods can be reused in various applications. Saves time and
costs!

19. Let’s connect in LinkedIn!
We deliver advice, skills and business opportunities in all machine learning playing fields.
Mircea Stoica
ML Engineer
Focus
AI | Reinforcement Learning
Data Science
Robotics
mircea @ heldenkombinat.com
HELDENKOMBINAT
MACHINE LEARNING
19