Video: https://youtu.be/Yv6L_b8ZrtU
Abstract:
Asthma, chronic pulmonary disease, is one of the major health issues in the United States. Given its chronic nature, the demand for continuous monitoring of patient’s adherence to the medication care plan, assessment of their environment triggers, and management of asthma control level can be challenging in traditional clinical settings and taxing on clinical professionals. A shift from a reactive to a proactive asthma care can improve health outcomes and reduce expenses. On the technology spectrum, smart conversational systems and Internet-of-Things (IoTs) are rapidly gaining popularity in the healthcare industry. By leveraging such technological prevalence, it is feasible to design a system that is capable of monitoring asthmatic patients for a prolonged period and empowering them to manage their health better.
In this thesis, we describe kBot, a knowledge-driven personalized chatbot system designed to continuously track medication adherence of pediatric asthmatic patients (age 8 to 15) and monitor relevant health and environmental data. The outcome is to help asthma patients self manage their asthma progression by generating trigger alerts and educate them with various self-management strategies. kBOT takes the form of an Android application with a frontend chat interface capable of conversing both text and voice-based conversations and a backend cloud-based server application that handles data collection, processing, and dialogue management. The domain knowledge component is pieced together from the Asthma and Allergy Foundation of America, Mayoclinic, and Verywell Health as well as our clinical collaborator. Whereas, the personalization aspect is derived from the patient’s history of asthma collected from the questionnaires and day-to-day conversations. The system has been evaluated by eight asthma clinicians and eight computer science researchers for chatbot quality, technology acceptance, and system usability. kBOT achieved an overall technology acceptance score of greater than 8 on an 11-point Likert scale and a mean System Usability Score (SUS) greater than 80 from both evaluation groups.
https://www.facebook.com/pg/Kno.e.sis/photos/?tab=album&album_id=2560068547361311
kBot: Knowledge-Enabled Personalized Chatbot for Self-Management of Asthma in Pediatric Population
1. 1
kBot - Knowledge Enabled Personalized
Chatbot for Self-Management of Asthma
in Pediatric Population
Committee Members:
Dr. Krishnaprasad Thirunarayan,
Dr. Valerie Shalin,
Dr. Maninder Kalra
Master’s Thesis Defense
Dipesh Kadariya
Kno.e.sis Center
Department of Computer Science and Engineering
Wright State University, Dayton, Ohio
June 24th
, 2019
Advisor:
Dr. Amit P. Sheth
kHealth
Ohio Center of Excellence in Knowledge-Enabled Computing
2. 2
Outline
● Introduction
○ Background
○ Challenges
○ Related work
● Methodology
○ Approach
○ Implementation
● Evaluation and results
● Conclusion and future work
3. Asthma
● Chronic pulmonary disease
● Multifactorial
● Can be serious (life threatening)
There is no cure for asthma, but it can be managed so you live a normal, healthy life.
Demographics:
● 26.5 million out of which 6.1 million are children
● Most chronic pediatric condition accounting for 13.8 million
missed school days each year
[Source : CDC]
3
4. Asthma self-management
4
Strategies to help people better manage their asthma, keep it well controlled and live life to
the fullest.
Key Challenges:
● Tracking asthma
○ Monitor patient’s day to day asthma condition
● Ensuring Medical adherence
○ Track and encourage patients towards medical
adherence
● Avoiding environmental trigger
○ Continuously monitor and alert environmental triggers
● Self-management education
○ Educate patients with self-management strategies
5. Tracking Asthma
Tracking signs and symptoms of asthma so as to
understand and manage patient asthmatic condition.
Know
control zone
track
Medication
PEF log
Environmental
triggers
Asthma
journal
Tracking
Asthma Tracking:
● Keep a journal (track symptoms)
● Tracking medication intake and inhaler
use
● A daily peak flow log (pulmonary
function)
● Know your zone
● Know your triggers
5
6. Medical Adherence
If patients know that their medication use is being monitored,
they are more likely to adhere.[1]
The implication of innovative techniques, such as telemonitoring of patients to:
● Educate patients better
● Interventions such as:
○ Informational interventions: audio visual education
○ Behavioural interventions: medication reminders
Main causes:
● Forgetfulness and/or carelessness.
● Inadequate training in the inhalation techniques.
● Lack of understanding
Adherence to medication in asthma is a well recognized challenge
6
7. Pollen from
trees and
grass
Cold
air
Air Pollution
(PM2.5)
Ozone O3
Mold
spores
Smokes
tobacco, fire
place
Animal danders
from hair, skin, or
feathers
Dust mites
feces
Cockroaches
feces
Asthma Triggers
Allergic Asthma - most common form of asthma
Variations of trigger:[1]
About 90% of pediatric asthma is due to allergies
7
8. Avoiding Triggers
Asthma -multifactorial disease
Two different asthma patients might react differently
to same asthma trigger
Personalized approach:
- Monitor day-to-day asthma of each patient
- Continuously monitor their environment
- Systematically rank the trigger based on patients
symptoms frequency
- Alert patient when triggers are present
Monitor Patients’
Asthma
Monitor Patients’
Environment
Potential Triggers
Alert when present
Personalized approach to avoid asthma triggers
8
9. Self-management education
Self-management education for children to improve coping with asthma in daily life.
Knowledge:
● Increase child’s knowledge and understanding of asthma and the
medicine
● Bring positive attitude towards need for medication
● Healthy behaviour, healthy diet, healthy environment
Self-management skills:
● Identify early signals and prevent asthma symptoms
● Act at an early stage of asthma symptom
● Relaxation techniques
9
10. kHealth for Asthma
10
kHealth: A knowledge-enabled semantic platform that captures
patient generated multimodal data and analyze it for actionable
insights.
Completed: 83
Consented: 107
- Continuous monitoring -> useful insights of patients health
- Poor medication adherence leads to poor asthma control
- different patients reacted to different environmental
triggers with varying intensity
Findings:
11. 11
Challenges
★ Poor patient compliance badly affects the quality and quantity of data
collection.
★ Poor medical adherence and improper inhaler use technique minimizes the
medical effectiveness.
★ Due to multifactorial nature of asthma, general treatment approach such as a
generic asthma care plan proves to be ineffective in most of the cases
demanding a more personalized approach.
12. Need...
A system to:
- Closely interact with patients
- Monitor patients’ wellbeing
- Encourage medical adherence
- Monitor patients environmental factors and
alert triggers
- Educate patients with self-management skills
Overall, a system that enables patients to self-manage
asthma and live their life to fullest.
12
13. How chatbot can help?
Creates a conversational environment:
- Available 24/7
- Natural
- Engaging
- Prompt response
Relevant content based on responses
Personalization
Easier to gain trust
Scalable
13
An asthma health
companion
14. Study Main objective Platform
Fadhil et al. (2017)
● Encourage adult population towards healthy eating habits to
prevent weight gain
Mobile- app
Your.MD
● Provide online health counseling to patients
● Help find online doctors
Web
MANDY (Ni et al.
2017)
● A primary care chatbot to assist healthcare staffs by automating
the patient intake process Mobile-app
FLORENCE
● Reminds patients to regularly take their medication
● Provides information related to their medicine
Facebook
Messenger
SMOKEY
● Alerts patients about air pollution in the city
● Regular air quality update
Facebook
Messenger
14
Technology in healthcare
15. 15
Technology/Application for Asthma
Study Main objectives Limitations Platform
Wella Pets
● Teach proper inhaler technique to asthmatic kids
● Help Identify trigger within house
● How to act during symptom
● No tracking of patients asthma
● Does not provide feedback to user
inputs
Mobile game
(Android/iOS)
Asthma Buddy
National Asthma
Council Australia
● Remind patients to take medication
● Track changes in symptoms
● Lack of Self-management
education
● No environmental alert
● No PEF tracking
Mobile App
(Android/iOS)Propeller app
Propeller health
● Automatically track medication use (where and
when)
● Alert doctor and family members on worsening of
asthma.
● Alert air quality issues
● Lack of Self-management
education
● No PEF tracking
AsthmaMD
● Log asthma symptoms in digital diary
● Asthma journal: Quick view of past asthma data
● Securely Share asthma record to physician
● Physician centric (directed to the
physician not to the clinician)
16. Thesis Statement
16
Smart conversational systems can continuously monitor patient’s relevant
health signals, including their medication adherence, and environmental data.
Through contextual and personalized processing of these data, it can enable
self-management of patient’s chronic disease. This work is presented in the
context of asthma.
Part of this work was published in SMARTCOMP 2019 workshop.
Dipesh Kadariya, Revathy Venkataramanan, Hong Yung Yip, Maninder Kalra, Krishnaprasad Thirunarayan, Amit Sheth. "kBot:
Knowledge-enabled Personalized Chatbot for Asthma Self-Management". In Proceedings of the IEEE SMARTSYS Workshop on Smart
Service Systems (SMARTCOMP 2019). IEEE, 2019.
18. 18
kBot Features
Monitor patients’ environmental factors and alert potential asthma triggers.
- Data watcher - periodically monitor weather data
- Mechanism to rank triggers based on symptom frequency
- Weather alert system
Monitor and encourage patients to improve medical adherence.
- Customizable Medication reminders
- Educate patients on importance of medicine to keep their asthma well controlled
Collect data on patient’s asthma for a given period of time in order to assess the control level.
- Refined and modified kHealth asthma questionnaire for simplicity and naturalness
- Ask questions in the form of conversation
Educate patients on asthma self-management skills.
- Skills on how to deal with common symptoms
- Relaxation techniques
- How to avoid common asthma triggers
19. Data collection
Asthma related data
Captured from user conversation
● Day/night symptom
● Controller medicine use
● Peak flow meter reading
● Current symptom
● Worsening symptom
● Rescue medicine use
19
}Daily Asthma
Questionnaire
}Report
Symptom
Environmental parameters
Collected from 3rd
party weather APIs
● Ozone
● Particulate matter 2.5
● Pollen index
● Temperature
● Humidity
Twice a day: morning and evening
Patient initiated reporting
20. Dialogue processing
20
I am experiencing cough
and wheeze
Intent: collect symptom
Entities: [Cough, Wheeze]
Context: current_symptom
User: KHB111
timeStamp: 01-02-2018
19:00:00
kBot
That’s not good…
Did you take your Albuterol
for the relief?
parse and process the user dialog.
★ Intent: Group of typical examples and logic that relates
to a common goal of the user.
★ Context: Set of conversation or fact around which
conversation is going on.
21. Outdoor Data Collection
21
Parameter Frequency
PM2.5 Hourly (24/day)
Ozone Hourly (24/day)
Temperature Every 2 hour (12/day)
Humidity Every 2 hour (12/day)
Pollen count Twice a day
Data Collection
Data watch
Collects outdoor parameters per zip code
from 3rd
party weather APIs at different
frequencies and commits to the cloud
storage
Constantly checks outdoor parameter values
collected by Data Collector and calls kBot
environmental alert service if encountered a
parameter in unhealthy range.
23. How kBot uses asthma domain knowledge to provide
contextually relevant information
Contextualization
23
Mayo clinic
https://www.mayoclinic.org/diseases-conditions/asthma/sympto
ms-causes/syc-20369653
Asthma and allergy foundation of America
https://www.aafa.org/
Verywellhealth
https://www.verywellhealth.com/asthma-overview-4582010
Asthma domain knowledge
24. 24
Medication and inhaler use
★ Medicine images
★ Inhaler use video
Use-inhalers.com
Video Content
Educate patient
for proper use of
inhaler
Medication images
Help patient
identify the
correct medicine
Domain Knowledge
Medication Image
American Association
for Respiratory Care
25. Personalization
25
Example
A snapshot of kBot Conversation
where kBot uses patient’s past data
and environmental data to provide
personalized alert to the patient.
A conversation without contextualization
and personalization provides a response
to the user question like a question
answering system.
Hi bot!
Hi there!
It looks like fairly
sunny outside. Enjoy
the day.
How’s the weather
today?
Thank you!
Personalization: Tailored future course
of action based on individual
characteristics of the patients such as
their health history, environment,
activity, and lifestyle.
Color coded personalized
alert based on patient health
condition.
26. 26
Initial user profile setup
➔ Patient ID (Anonymized)
➔ Age
➔ Deployment period (date range)
➔ Gender
➔ Zip Code
➔ Clinician contact point
➔ Prescribed rescue medicine(s)
➔ Prescribed controller medicine(s)
Patient consent and deployment
User
Index
... ...
Update
Patient
Use patient’s past
data to generate
personalized
response.
Cloud Database
{...
Patient data
...}
kBot
★ Symptom trend
★ Medication list
★ Medication compliance
★ Personalized triggers
★ Environmental data
27. Environmental Alert
27
Data watcher
Notification
Service
Trigger_list:
{
R1: pollen,
R2: Ozone,
...}
Patient specific
Ranked trigger list
Ranking: Frequency of co-occurrence with the symptom within 48
hour window.
[Shortness of
Breath]
5:00 PM
[Pollen] > 5
9:00 AM
1 Instance of
Co-occurrence
HTTP POST
{zip, trigger }
Personalized
Environmental Alert
Push Notification()
Patients:
Zip = affected zip code
and
trigger in [Trigger_list]
Trigger Frequency of
co-occurrence
Pollen 4
Ozone 2
Temperature 1
July 24th July 23rd
Example:
Detects
environmental
parameters in
unhealthy range
28. 28
PEF and Asthma Zone
Personal best PEF: Best PEF value out of 3 initial consecutive readings
Digital
Peak Flow Meter
PEF<50% 50%<PEF<80% PEF>80%
Zones based on PEF
Frequency: Twice a day
PEF
reading
Personal
best
Asthma
zone
If
Red/Yellow Alert
Patient
Compares reported PEF with
patients personal best
29. 29
Technical Requirements
★ Continuously monitor patient’s health and environment data.
★ Process these data in near real-time
★ Generate prompt response
★ Scalable over growing end user and their data
★ Secure data processing and exchange
31. Services
● Push Notification
-generate notification and alert
● Weather
-weather report and environmental data
● Data collector
-capture data from user text
● Email
- Send out email
● FileIO
- Log raw conversation
● Elastic
-Database operations
31
32. Data Exchange Security
32
1010...1...111..011.....1110.....0
Client Server
HTTP/1.1
Kno.e.sis private cloud
database
Socket.IO
Socket.IO (web-socket)
- Real-time data exchange
- Persistent connection
- Continuous data stream
Secure Socket Layer(SSL) certificate
- Binds cryptographic key
- HTTPS connection
- Secure connection between intended
parties
- Encrypted data transmission
HTTP connection
- Create, Read, Update,
Delete (CRUD)
Encrypted Channel
33. Cloud Database
- Database + full-text search engine
- Apache Lucene library
- HTTP web interface
- Schema-free JSON documents (noSQL)
- Dynamic mapping
- Open Sourced
Engineered for performance and scalability
33
37. Preliminary evaluation for acceptance of the technology as a tool to self-manage
asthma.
Population: 8 asthma clinicians (domain experts) and 8 computer science researchers
(non-domain experts)
Evaluation criteria: chatbot performance, technology acceptance and system usability
Naturalness
● Dialogue
naturalness
● Dialogue ambiguity
● Dialogue simplicity
Interpretability
● User intent in the
conversation
● User ability to express
their asthmatic
condition through
conversation
Information
Delivery
● Right information at
right time
● Information relevant
to asthma care.
37
38. 38
Metrics Questions
Quality of
chatbot
Naturalness
● kBot uses simple and understandable vocabulary.
● kBot dialogues were unambiguous.
● kBot dialogues were natural.
Information delivery
● kBot provides patients with the right information at right time.
● Information provided by kBot helps an asthma patient manage their
asthma better.
Interpretability
● kBot properly understood what a patient intended to say during the
conversation.
● The patients will be able to express their current asthma condition and
medication usage accurately through the conversation.
Technology acceptance
● The information kBot is trying to collect through the conversation
adequately conveys a patient’s asthma condition.
● I recommend this technology to monitor and manage a patient’s daily
asthma condition.
● Overall, I am very satisfied with this technology.
Evaluation Metrics
39. Evaluation result
39
11 point Likert Scale
Range: 0-10
0 - Strongly Disagreed
10 - Strongly Agreed
Total 16 responses
8 - Clinicians
8 - Researchers
Mean >= 8.25
40. System Usability Scale (SUS)
40
➔ Standard 10 item questionnaire
➔ 5 point likert-scale (ranging 0-4)
➔ Evaluates usability of wide variety of
systems
➔ Can be used for small sample sizes
➔ The final SUS score range from 0-100.
kBot achieved a SUS score better than 82
from both the evaluation population.
> 68 (above
Average)
41. Conclusion
41
- As a chronic multifactorial disease, assessing and managing
asthma within the traditional clinical setup has always been
challenging.
- Technology such as chatbot can engage with patients in their
day-to-day life monitoring and helping them manage their
asthma.
- Use of in-depth contextual knowledge and personalized
approach increasing its effectiveness and achieve better
compliance.
42. 42
Future work
★ Incorporate IoT sensors to collect additional
patient data such as indoor air quality, sleep and
activity to get better insights into patients health.
★ Design a custom language model and train it on
real-life patient-doctor conversation data to
deliver a more natural and human like
conversation.
★ Connect kBot with kHealth Dashboard enabling
clinician to have real-time insights into patients
health and decision making.
45. 1. Dipesh Kadariya, Revathy Venkataramanan, Hong Yung Yip, Maninder Kalra, Krishnaprasad Thirunarayan, Amit Sheth. "kBot:
Knowledge-enabled Personalized Chatbot for Asthma Self-Management". In Proceedings of the IEEE SMARTSYS Workshop on Smart Service
Systems (SMARTCOMP 2019). IEEE, 2019.
2. Revathy Venkataramanan, Krishnaprasad Thirunarayan, Utkarshani Jaimini, Dipesh Kadariya, Hong Yung Yip, Maninder Kalra, Amit
Sheth. Determination of Personalized Asthma Triggers from Multimodal Sensing and Mobile Application.,JMIR Pediatr Parent
(forthcoming). doi:10.2196/14300
3. Utkarshani Jaimini, Krishnaprasad Thirunarayan, Maninder Kalra, Revathy Venkataramanan, Dipesh Kadariya, Amit Sheth, "How Is My Child's
Asthma?" Digital Phenotype and Actionable Insights for Pediatric Asthma., JMIR Pediatr Parent 2018;1(2):e11988, DOI: 10.2196/11988.
4. Vaikunth Sridharan, Revathy Venkataramanan, Dipesh Kadariya, Krishnaprasad Thirunarayan, Amit Sheth, Maninder Kalra.
"Knowledge-enabled Personalized Dashboard for Asthma Management in Children". In proceedings of the American College of Allergy, Asthma &
Immunology Annual Meeting(2018).
5. Amit Sheth, Hong Yung Yip, Utkarshani Jaimini, Dipesh Kadariya, Vaikunth Sridharan, Revathy Venkataramanan, Tanvi Banerjee, Krishnaprasad
Thirunarayam, Maninder Kalra. "Personalized Health: Using Semantically Integrated Multimodal Data for Patient Empowered Health Management
Strategies". Technology Showcase, National Institute of Health - June 2018.
6. Utkarshani Jaimini, Hong Yung Yip, Revathy Venkataramanan, Dipesh Kadariya, Vaikunth Sridharan, Tanvi Banerjee, Krishnaprasad
Thirunarayam, Maninder Kalra, Amit Sheth. "Digital Personalized Healthcare technology for Pediatric Asthma". mHealth Technology Showcase,
National Institute of Health - June 2018.
7. Amit Sheth, Hong Yung Yip, Utkarshani Jaimini, Dipesh Kadariya, Vaikunth Sridharan, Revathy Venkataramanan, Tanvi Banerjee, Krishnaprasad
Thirunarayam, Maninder Kalra. "Feasibility of Recording Sleep Quality And Sleep Duration Using Fitbit in Children with Asthma". Abstract in the
32nd Annual Meeting of the Associated Professional Sleep Societies (SLEEP) , 2-6 June 2018, Baltimore, MD.
8. Amit Sheth, Tanvi Banerjee, Utkarshani Jaimini, Dipesh Kadariya, Vaikunth Sridharan, Krishnaprasad Thirunarayam, Revathy Venkataramanan,
Hong Yung Yip, Maninder Kalra. "Correlating Multimodal Signals With Asthma Control In Children Using kHealth Personalized Digital Health
System". Abstract in American Thoracic Society (ATS) International Conference , 18-23 May 2018.
Publications
45
46. This research is partially supported by National Institute of Health(NICHD/NIH) under
the Grant Number: 1R01HD087132. Any options, findings, and conclusions or
recommendations expressed in this work are those of the author(s) and do not
necessarily reflect the official views of the National Institute of Health.
46
47. References
47
1. CDC, “Center for Disease Control and Prevention-Most recent asthma data,”
https://www.cdc.gov/asthma/most_recent_data.htm, 2018, [ accessed March 8-2019].
2. https://www.lung.org/lung-health-and-diseases/lung-disease-lookup/asthma/learn-about-asthma/what-is-asthma.html
3. K. Sumino and M. D. Cabana, “Medication adherence in asthma patients,” Current opinion in pulmonary medicine, vol. 19, no. 1,
pp. 49–53, 2013.
4. A. Whittamore, “Technology can revolutionise supported self-management of asthma,” Primary Care Respiratory Society UK vol.
4, 2017.
5. A. Fadhil and S. Gabrielli, “Addressing challenges in promoting healthy lifestyles: the al-chatbot approach,” in Proceedings of
the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. ACM, 2017, pp. 261–265.
6. L. Ni, C. Lu, N. Liu, and J. Liu, “Mandy: Towards a smart primary care chatbot application,” in International Symposium on
Knowledge and Systems Sciences. Springer, 2017, pp. 38–52.
7. R. Shah and L. Velez, “Effectiveness of software-based patient education on inhaler technique: A clinical study,” European
Respiratory Journal ,vol. 44, no. s58, 2014.
8. Dayer, L., Heldenbrand, S., Anderson, P., Gubbins, P. O., & Martin, B. C. (2013). Smartphone medication adherence apps: potential
benefits to patients and providers. Journal of the American Pharmacists Association : JAPhA, 53(2), 172–181.
doi:10.1331/JAPhA.2013.12202
9. https://www.webmd.com/asthma/allergic-asthma-what-is-it#1
10. http://www.rtmagazine.com/2007/02/self-monitoring-asthma-at-home/
11. Colland VT. Learning to cope with asthma: a behavioural self-management program for children. Patient Educ Couns. 1993 Dec
31;22(3):141-52. PubMed PMID: 8153036.
48. Resources used
48
1. Asthma background:
https://www.dreamstime.com/asthma-patient-girl-inhaling-medication-treating-shortness-o-asthma-patient-girl-inhaling-me
dication-treating-shortness-image102477568
2. Self-management background: https://freerangestock.com/photos/48233/happy-child-drawing-a-sunny-landscape-.html
3. Medical adherence background: https://www.freepik.com/free-photo/pills-forming-heart_3377211.htm
4. Chatbot background:
https://www.freepik.com/free-vector/artificial-intelligence-isometric-ai-robot-mobile-phone-screen-chatbot-app_3629606.ht
m
5. Icons: https://icons8.com/
49. Acknowledgement
49
Dr. Amit Sheth
(Advisor)
Dr. Krishnaprasad
Thirunarayan
Dr. Valerie Shalin Dr. Maninder
Kalra
Committee members:
Revathy
Venkatramanan
(Collaborator)
Shreyansh Bhatt Kirill Kultinov Vaikunth
Sridharan
50. kHealth Team
50
Dr. Amit Sheth
(Principal investigator)
Dr. Krishnaprasad
Thirunarayanan
(Co-investigator)
Dr. Maninder Kalra
Co-investigator
(Pulmonologist at Dayton Children’s
Hospital)
Revathy
Venkatramanan
Utkarshini Jaimini Hong Yung Yip Kirill Kultinov
Dr. Tanvi Banerjee
(Co-investigator)
Faculty Clinical Collaborator
Graduate Students