Hironobu Takagi, IBM Research, Tokyo presented "Cognitive Assistance for the Aging" as part of the Cognitive Systems Institute Speaker Series.

1. IBM Research - Tokyo
Eldercare Research
Hironobu Takagi
IBM Research – Tokyo
Aging Strategic Initiative

2. IBM Research - Tokyo
PercentageofPopulation65yearsandolder
Japan
Italy
Germany
Ireland
China
Australia
Brazil
US
India
Egypt
2017
2017
27%
World Population Aging

3. IBM Research - Tokyo
62 87
28%
Dementia 15% + MCI 13%
estimation by Japanese Government, 2012
of people over the age of 65
are at the risk of dementia.
70%of people want to work over
the age of 70 (and forever).
Life expectancy of women in
1950 and 2015.
* numbers in Japan
3%
Total social cost per year in
Japan is $145B or
of the GDP

4. IBM Research - Tokyo
We need a new model for
active life-long society.

5. IBM Research - Tokyo
Eldercare Research: Focus Areas
Social
Intelligent assist
Sensing
Accessibility
Brain
2003 - 2012
2010 -
2015 -
2016 -
Physical assist
Japan Post
Japan Post

6. IBM Research - Tokyo
Eldercare Pilot Project with Japan Post
• Pilot with 1000 elderlies completed (Oct. 2015 – Oct. 2016)
• Medicine reminders, video telephony, online shopping, photo sharing, etc.

7. IBM Research - Tokyo
Global Apple + IBM Apps for Elder Services
Care givers
Family
members
Elderly
people

8. IBM Research - Tokyo
Italian Post

9. IBM Research - Tokyo
Eldercare Research: Focus Areas
Social
Intelligent assist
Sensing
Accessibility
Brain
2003 - 2012
2010 -
2015 -
2016 -
Physical assist
Japan Post
Japan Post

10. IBM Research - Tokyo
Voice-base natural user interface for daily watchover
• Analyze life patterns, feelings, interests and issues from daily conversation (e.g. cognitive decline detection)
• Share information with families and care givers
• Provide support that can keep the elderly self-sufficient
Conversation as Sensors
User
Weather
Events
Meals
Activities
Life Patterns
Cognitive issues
Feelings
Interests
Family & Friends
Daily Engagement Alerts & Reports
Support workers
WATSON

11. IBM Research - Tokyo
11
Conversational Eldercare - DEMO

12. IBM Research - Tokyo
Questionnaire Intent
“What did you have last night?” Dementia (Neuro Cognitive Disease) Risk
Assessment
“Did you go out today?” Activity checking
“You mentioned about gardening in diary. Are
you interested in community gardening?”
Recommendation of activities
”Which city do you want to travel?” Marketing
“The shop, you mentioned last week, is now on
sale!”
Promotion
“We delivered a package, but you were not
there. Please contact post office"
Notification (business related)
Intent of each questionnaire

13. IBM Research - Tokyo
Emotion and Physical Condition Recognition
Voice data Features
988 acoustic features for
emotion recognition.
• Intensity
• Loudness
• 12 MFCC
• Pitch (F0)
• Probability of voicing,
F0 envelope,
• 8 LSF,
• Zero-Crossing Rate
Labeled
voice
dataset
ML-based
Regression
Estimation
Feeling good
or bad
40
0
100
Physical
condition
Ex: Good night
*Each circle represents one greeting.
*One participant data
Emotion
NegativePositive
Confusion FrustrationDelight Flow
Physical Condition
Emotion
Yorktown – Speech team
Tokyo Research
Happy
Sad

14. IBM Research - Tokyo
Dashboard for Family and Care Givers
Condition
Emotion
(voice sound
analysis)
Personality analysis
(Watson personality
insights)
Daily activities
Eating habits
Dementia
questionnaire
scores
and more ….

15. IBM Research - Tokyo
Eldercare Research: Focus Areas
Social
Intelligent assist
Sensing
Accessibility
Brain
2003 - 2012
2010 -
2015 -
2016 -
Physical assist
Japan Post
Japan Post

16. IBM Research - Tokyo
th Leading Cause of Death
6
But, interventions during MCI have been
shown to be effective
Of the top 10 killers, AD is the only one that
cannot be prevented, cured, or even slowed
50-95%
of the people with dementia never have screening
• High income countries: 40-50% (e.g. US 45%)
• Low/Middle income countries: 5-10%
Societal Impact of Dimentia
Poor gait performance can be used for detection and also
prediction of dementia and mild cognitive impairments before
up to 7-12 years
[Buchman & Bennett, Expert Rev. Neurother., 2011]
[McGough EL et al . Phys Ther. 2011; Beauchet O et al., J Nutr Health Aging. 2013]

17. IBM Research - Tokyo
• Gait speed & its variability
• Step frequency
• Stride time variability
• Step-length & its variability
• Foot swing velocity
• Stance and stride time
• Motor function
• Fall risks
• Cognitive decline
• Episodic memory
• Executive function
• Diseases
• Parkinson disease
• MCI
• Alzheimer’s disease
Feature extraction of gait
Features Detection

18. IBM Research - Tokyo

19. IBM Research - Tokyo
Brain Simulation: Understanding Alzheimer Disease (2016)
19
APOE-4 carriers
APOE-4 non-carriers
Investigate whether and how an abnormality in inter-areal structural connectivity observed in APOE-4
carriers (major risk factors in Alzheimer Disease) affects cortical information processing
Structural connectivity of APOE-4 carriers might
reduce cortical information propagation and lead to
negative effects in information integration.

20. IBM Research - Tokyo
Social
Intelligent assist
Sensing
Accessibility
Brain
Physical assist
Eldercare Research: Focus Areas

21. IBM Research - Tokyo
Thank you
Hiro Takagi
takagih@jp.ibm.com
Twitter: @hirotakagi