Machine Learning Methods in Detecting Alzheimer’s Disease from Speech and Language Alzheimer's disease affects millions of people worldwide, and it is important to predict the disease as early and as accurate as possible. In this talk, I will discuss development of novel ML models that help classifying healthy people from those who develop Alzheimer's, using short samples of human speech. As an input to the model, features of different modalities are extracted from speech audio samples and transcriptions: (1) syntactic measures, such as e.g. production rules extracted from syntactic parse trees, (2) lexical measures, such as e.g. features of lexical richness and complexity and lexical norms, and (3) acoustic measures, such as e.g. standard Mel-frequency cepstral coefficients. I will present the ML model that detects cognitive impairment by reaching agreement among modalities. The resulting model is able to achieve state of the art performance in both supervised and semi-supervised manner, using manual transcripts of human speech. Additionally, I will discuss potential limitations of any fully-automated speech-based Alzheimer's disease detection model, focusing mostly on the analysis of the impact of a not-so-accurate automatic speech recognition (ASR) on the classification performance. To illustrate this, I will present the experiments with controlled amounts of artificially generated ASR errors and explain how the deletion errors affect Alzheimer's detection performance the most, due to their impact on the features of syntactic and lexical complexity.

1. Machine Learning Methods in
Detecting Alzheimer’s Disease
from Speech and Language

2. Alzheimer’s Disease (AD)
●
●
●
●

3. Winterlight’s Assessment Tools
Placeholder for the video

4. Extracting Features from Speech
●
●
●
the boy is handing the girl &uh cookies and she's telling
him to be quiet i guess.

5. Consensus Networks: Motivation
●
○
○
●

6. Consensus Networks: Framework
Z. Zhu, J. Novikova, and F. Rudzicz. Detecting cognitive impairments by agreeing on interpretations of linguistic features. In Proceedings of NAACL,
2019

7. Consensus Networks: Algorithm

8. Model Characteristics

9. Benchmarks and Semi-supervised Learning
Z. Zhu, J. Novikova, and F. Rudzicz. Semi-supervised classification by reaching consensus among modalities. In: NeurIPS Workshop on
Interpretability and Robustness in Audio, Speech, and Language IRASL, Montreal, 2018

10. Effect of Automatic Speech Recognition
Placeholder for the video

11. ASR in Winterlight’s Assessment

12. Effect of ASR Errors on Classification
J. Novikova, A. Balagopalan, K. Shkaruta and F. Rudzicz. Lexical Features Are More Vulnerable, Syntactic Features Have More Predictive Power.
In: The 5th Workshop on Noisy User-generated Text at EMNLP, Hong Kong, 2019
●
●

13. Take Away
●
●
●
●
●
●
●
●

14. Thank you!
Questions?