1. Data Fusion of Multiple Physiological Sensors
for Use in Machine Learning
Computer Science Department
Professor Hirshfield and Leanne Hirshfield
Eseosa Asiruwa‘18
Matt Goon ’18
Mitchel Herman‘19
Sindy Liu ’18
GOALS
• Create a Python program that combines data from multiple
physiological sensors and runs the combined data through
machine learning algorithms
• Create a graphical user interface for the program in order to make
it easy to use
• Make the program flexible to the needs of the user by adding
options to customize on the code runs
INTRODUCTION
AcknowledgementsBen Sklar ‘18
Russell Glick ’18
Diane Paverman
Eric Murray
Mykhalio Antoniv ‘17
Eric Collins ’17
Languages and
Software Used
Python 2.7
Orange
BioSPPy
Numpy
SciPy
Pandas
Further Research
GUI
PROJ
MACHINE LEARNING
Machine Learning (ML) is a a form of artificial intelliegences that helps computers learn without being
explicitly programmed. Machine learning focuses on the development of computer programs that can
teach themselves to grow and change when exposed to new data. Machine learning uses data to
dectect patterns and adjusts program actions accordingly.
Our end goal: Create a program that takes advantage of pre-existing ML algorithms and will
continously run on a given set of large data and condition files. The program will be flexible enough
to be used for future experiments with minimal change. Credit: Techtarget.com
Machine learning algorithms have proven useful for
identifying complex patterns through the analysis of
large quantities of data and predicting outcomes from those
patterns. Past research projects used machine learning
algorithms to analyze data from a single physiological
sensor to predict the psychological state of the test subjects.
The program we created in the Python programming language
accepts data from five physiological sensors (functional near
infrared spectroscopy, electroencephalogram, respiration,
electrocardiogram, electrodermal activity) and creates files
that get“fused”so that all information can be processed by a
sophisticated, Python-based machine learning program,
Orange Data Mining.
Creating a graphical user interface for the program proved to be a second concurrent
goal for the project. This included customization options that affect how the program runs.
We expect that more research
will be done on top of the work
we have been able to accomplish.
In the future, since we made sure
that our code was flexible and easy
to understand, there is possibility
for the addition of more
pyschological sensors. There is
also the prospect of added
features for the senors that are
currently in the program. Further
additions to GUI would also be
possible in order to achieve
a better user experience.
Window 1
Select which
sensor(s) you
want to analyze
data from
Window 2
Select how you
want to format
the data (ROI, Z-
Score, Sampling
Rate)
Window 3
Select if you
want to include
SAX features. If
you do, input
the parameters
for SAX
Window 4
Input which conditons to be
analyzed and machine learning
paramters
Also choose which files you want to
output and the extension for
machine learning output
Window 5
Upload the data
files you want to
run machine
learning on
Proj.py can: Create .arff files for each subject for five physiological sensors
Create .tab files for each subject for five physiological sensors
Fuse .arff files for all the physiological sensors for each subject
Create a concatenated arff file for all the subjects
Run a variety of machine learning algorithms on .arff files
Proj.py is the main code we designed for the summer. The project can perform in a variety of different ways. Proj.py
works by formatting and passing data and condition files for all of the desired sensors through another program
called Arffgen. Arffgen then creates a .arff file which is used by the Orange ML algorithms. Proj.py then fuses the .arff
files for all the sensors and runs the fused data through different machine learning algorithms, creating ML files for
each subject.
Proj.py is set up to work in any way the user needs it to run. It can stop after any stop of skip steps in order to save time.
It is also completely expandable and should be able to work with other physiological sensors with minor changes to
the code.