Takashi Yoshinaga is a researcher at the Institute of Systems, Information Technologies and Nanotechnologies who is researching applications of augmented reality (AR) technology, particularly for medical ultrasound imaging. He has developed two AR systems to help inexperienced physicians learn ultrasound probe techniques by visualizing a skilled physician's prior probe movements and angles. He is also exploring collaborative AR systems using multiple devices and sensor data integration. His current work involves prototyping gesture control of HoloLens content using EMG sensors and sharing AR experiences across different devices in real time.

1. Takashi Yoshinaga
Institute of Systems, Information
Technologies and Nanotechnologies
(ISIT)
@ MTA SZTAKI, Hungary, Aug. 8th, 2018

2. • About Me
• Research
• Current Experiences with AR

3. • About Me
• Research
• Current Experiences with AR

4. Name:
Takashi Yoshinaga
Title:
Researcher
Affiliation:
Institute of Systems, Info. Tech. and Nanotech.
Works:
Daily life support with Augmented Reality (AR)

5. • About Me
• Research with AR for Echography
• Current Experiences with AR

6.  Medical image diagnosis technology using ultrasound.
 Real-time image of internal organ is obtained just by putting
probe on a patient’s body.
Probe

7.  Field of view of echogram is not wide.
 Spatial relationship between internal organ and cross
section must be understood by physician to get echogram.
Cross section

8. We have proposed 2 kinds of AR systems which
can visualize probe operation of skilled physician.
(1) Probe position/angle recorded in previous.
(2) Real-time instruction from remote expert.
Probe
AR Interface
Rotate
Move

9. Green plane is probe operation
recorded by skilled physician
Unskilled physician could acquire good echogram by referring
to superimposed position/angle of probe recorded in previous.
https://youtu.be/Dpuc243qCzw

10. Skilled Physician
Instruction via GUI
Patient’s Side
Non-verbal communication can help remote assistance.
https://youtu.be/WV99AmD-A0E

11. Using HoloLens & markerless tracking tech for AR echography.
https://youtu.be/zI8S49qpiOs

12. • About Me
• Research with AR for Echography
• Current Experiences with AR

13.  Interested in collaboration system among AR
devices and sensing devices.
 Now prototyping some kinds of systems.
Sensing Devices
Data
Server Machine
AR Devices
Data
Data

14. EMG(Electromyography) sensor is used for
adding gesture input to HoloLens content.
https://youtu.be/b0IcyAj3ag0

15. AR visualization remote environment scanned by Kinect
https://youtu.be/E9v58QAFfNg

16.  The same scene is shown in different devices.
 Operation of a user is shared with others.
https://youtu.be/W9fVF-aoc5w

17. Takashi Yoshinaga