A mHealth solution for contact-less self-monitoring of vital sign parameters

A mHealth solution for contact-less
self-monitoring of vital sign parameters
Gabriella Casalino
gabriella.casalino@uniba.it

A mHealth solution for contact-less self-monitoring of vital signs parameters
http://www.di.uniba.it/~cilab

AI 2.0-Enabled Next Generation Intelligence of Things for Smart Enterprise Systems", Computers, Materials & Continua, IF. 4.89

Prof. Giovanna
Castellano
Gianluca
Zaza
Vincenzo
Pasquadibisceglie

Self-monitoring

Motivation
Cardiovascular
disease
Contact
devices

Monitoring techniques
Pulse Oximeter Smartphone Webcam

Solution
Development of a smart mirror capable of
detecting vital parameters (HR, RR, SpO2, lip
color) and predicting cardiovascular risk level.
How
Why
Using non-contact sensors, i.e. a webcam.
What
To encourage the culture of prevention and
provide a tool for self monitoring.

Photoplethysmography

Solution - key features
Cheap Cross-Platform
Real-time video
and Contactless

Vital parameters
measurement pipeline
Face
detection
ROI
detection
ROI
tracking
Vital
parameters
estimation
Video
frame
Vital
parameters

Face detection
The patient stands in front of the camera at a distance
of about 50 cm.
• Real-time capture of 24-bit RGB video frames;
• Resolution of captured channels is 480 * 200 pixels;
• Acquisition of the first 26 sec to remove the initial camera distortion;
• The following frames are captured every 2 sec.
• The system was developed completely in Python.

ROI detection
Regions of Interest
1. Forehead of size 90 x 30 pixel;
2. Right cheek of size 20 x 30 pixel;
3. Left cheek of size 20 x 30 pixel;
These ROIs are related to areas that are rich in remote
photoplethysmography signal.

ROI Tracking
• To reduce motion artifacts due to
head movements we have applied the
Kanade-Lucas-Tomasi (KLT)  
tracking method.
• This method creates a matrix of face movement
between subsequent frames. Coordinates of ROIs in one
frame are transformed into new coordinates in the next
frame according to the movement of the face.
• A further ROI is considered at the center of the face of
size 75 x 80 pixel.

Vital parameters estimation
Following step is the analysis of the RGB signals extracted from the ROIs
marked in each video frame of a short sequence in order to estimate the
oxygen saturation in blood.
1. Each ROI is separated into RGB channels and each channel is averaged
over all the pixels of the ROI;
2. For each ROI, the average signals (VR, VG, VB) are collected on the N
frames of the sequence, therefore we obtain a final signal matrix VRGB with
3 N dimensions;

3. The VRGB signals are improved through a preprocessing phase:
• Finite Impulse Response (FIR);
• Chrominance-based method;
• Linear interpolation;
• Power spectral density (PSD).
4. Vital parameters are estimated.

Cardiovascular risk prediction

Fuzzy Inference System

Fuzzy variables
Input linguistic variables:
Heart rate [20-180] bpm
Linguistic terms:
- Bradycardia
- Normal
- Tachycardia
Breathing rate [0-80] rpm
Linguistic terms:
- Hypopnea
- Normal
- Tachypnea

Fuzzy variables
Input linguistic variables:
Blood Oxygen Saturation [75-100] %
Linguistic terms:
- Critical
- Low
- Normal
Lip color [0-16]
Linguistic terms:
- Regular
- Medium
- Irregular

Fuzzy variables
Output linguistic variable:
Risk level [1-4]
Linguistic terms:
- Low: 1
- Medium: 2
- High: 3
- Very High: 4

Fuzzy rules
Guidelines given by the clinician
1 - When all the vital signs exhibit standard values, the risk level is low;
2 - When one vital sign exhibits a nonstandard value, the risk level is
medium;
3 - When two vital signs exhibit some nonstandard values, the risk level
is high;
4 - When three vital signs exhibit some nonstandard values, the risk
level is very high.

Fuzzy Rules Base

Future works
Detection of vital
parameters using
other devices
Development of new
methods for vital
parameters
measurements

Future works
Development of a
telemedicine
platform
Development of
different prediction
systems based on ML
and DL techniques

A mHealth solution for contact-less self-monitoring of vital sign parameters
References
• Casalino, G., Castellano, G., Zaza, G., (2020) A mHealth solution for contact-less self-monitoring of blood oxygen saturation, in Proc. of
ICTS4eHealth Workshop in conjuction of the 25 IEEE Symposium on Computers and Communications (ISCC2020).
• G. Casalino, G. Castellano, V. Pasquadibisceglie, G. Zaza, (2018) Contact-less real-time monitoring of cardiovascular risk using video imaging and
fuzzy inference rules, Information, 2019, Vol. 10, Issue 1, pp. 2078-2489, 10.3390/info10010009
• G. Casalino, R. Grassi, M. Iannotta, V. Pasquadibisceglie and G. Zaza, (2020) A Hierarchical Fuzzy System for Risk Assessment of Cardiovascular
Disease 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), pp. 1-7, doi: 10.1109/EAIS48028.2020.9122750.
• Casalino G., Castellano G., Pasquadibisceglie V., Zaza G., (2020) Evaluating end-user perception towards a cardiac self-care monitoring process.
G. M. P. O’Hare et al. (Eds.): MobiHealth 2019, LNICST320, pp. 1–17, 2020, DOI:10.1007/978-3-030-49289-2_4.
• Zaza, G. (2019) Monitoring Cardiovascular Risk by Video Processing and Fuzzy Rules. Discussion and Doctoral Consortium papers of AI*IA 2019 -
18th International Conference of the Italian Association for Artificial Intelligence.
• Casalino G., Castellano G., Castiello C., Pasquadibisceglie V., Zaza G. (2019) A Fuzzy Rule-Based Decision Support System for Cardiovascular
Risk Assessment. In: Fullér R., Giove S., Masulli F. (eds) Fuzzy Logic and Applications. WILF 2018. Lecture Notes in Computer Science, vol 11291.
Springer, Cham
• Pasquadibisceglie, V.; Zaza, G.; Castellano, G. A personal healthcare system for contact-less estimation of cardiovascular parameters. Proc. of the
AEIT International AnnualConference (AEIT2018), 10.23919/AEIT.2018.8577458.
• Castellano, G.; Castiello, C.; Pasquadibisceglie, V.; Zaza., G. FISDeT: Fuzzy Inference System Development Tool. Int. J. Comput. Intell. Syst.
2017,10 , 13–22, doi:10.2991/ijcis.2017.10.1.2.

A mHealth solution for contact-less self-monitoring of vital sign parameters
Thanks
https://sites.google.com/site/cilabuniba/people/gabriella-casalino
{gabriella.casalino, giovanna.castellano,
vincenzo.pasquadibisceglie, gianluca.zaza}@uniba.it

A mHealth solution for contact-less self-monitoring of vital sign parameters

Recommandé

Recommandé

Contenu connexe

Similaire à A mHealth solution for contact-less self-monitoring of vital sign parameters

Similaire à A mHealth solution for contact-less self-monitoring of vital sign parameters (20)

Plus de Gabriella Casalino

Plus de Gabriella Casalino (13)

Dernier

Dernier (20)

A mHealth solution for contact-less self-monitoring of vital sign parameters