CHI2021 LBW

1. From Virtual to Real World: Applying
Animation to Design the
Activity Recognition System
Keio University
Chengshuo Xia, Sugiura Yuta
Late Breaking Work

2. • Data-driven Human Activity Recognition (HAR) system relies on
the training dataset which is collected from the significant signal
variation regions.
• Such regions limited the flexibility to alter the sensor position.
• Change the position requires the system to re-collect the sensory
data from desired position.
2
Background
Specific position
sensory data
collection
HAR Training Application

3. • A virtual platform combined with sensor simulation to design the
non-visual HAR systems.
• Provide the accuracy as the feedback of examined positions via
cross-validation.
• Virtual sensor can be tested to check the corresponding accuracy.
• Classifier interface trained by simulation signal can be generated
forwarding to the real activity recognition.
3
System

4. UNITY 3D based platform
• Input data stream:
• Generated from Xsens MVN taken by subjects.
• Sensor Simulation:
• HAR system based on virtual dataset:
• Down-sampled to decrease frame rate influence
• Segmentation
• Augmentation
4
System

5. • Case-a:
Wearable acceleromters system for daily activity recognition
Testing & Application
Several types of daily activity recognition with an accelerometer
worn on the body.
User can have customized demand of sensor position and
activity recognized.

6. • Case-b:
Wearable distance sensor system for exercises recognition
Testing & Application
Design a wearable distance sensor system which the sensor is
attached on the lower limbs of an individual.

7. • Case-c:
Distance sensing system for motion recognition in a bathroom
Testing & Application
Distance map into
grayscale figure

8. • Result:
Testing & Application

9. 9
Conclusion
• Virtual simulation environment to produce the simulated multi-
modal sensor data.
• Drop back the position choice to the end-user side.
• Arbitrarily to place the sensor in the virtual scenario.
• Combine with virtual sensory, improve the process of HAR
system design.
• Recognize the activity in the real-world
• Future work:
• Virtual motion generator
• VR/AR combined for serious/somatosensory game

10. Thank you very much!