Haptic Communications

Technische Universität München

Haptic Communications
Fernanda Brandi, Rahul Chaudhari, Burak Cizmeci, Julius Kammerl,
Clemens Schuwerk, Eckehard Steinbach, Xiao Xu
Institute for Media Technology, TU Munich
Sandra Hirche, Iason Vittorias
Institute of Automatic Control Engineering, TU Munich

F

t
Klagenfurt University, March 27, 2012


The quest for immersive communication: telepresence

Network

audiovisual
communication

Although conversational services are bidirectional,
audiovisual data communication is 2x unidirectional

IEEE Signal Processing Magazine, vol. 28, no. 1, January 2011
Special Issue on Immersive Communication
Guest editors: Y. Altunbasak, J. Apostolopoulos, P. Chou, and B. H. Juang

27.03.2012 Eckehard Steinbach et al. 2


Telepresence + Haptics = Teleaktion (Telemanipulation)

Local Control Loop Local Control Loop

Sensors
&
Network
Actuators

Operator with audio-visual-haptic Teleoperator in
Human-System-Interface communication remote environment

Operator performance increases significantly in telemanipulation of
remote objects when haptic feedback is provided
Haptic communication is by definition bidirectional
[Cohen Loeb 1983; Hannaford et al. 1993; Hirzinger et al. 1994; Srinisavan et al. 1997;
Dennerlein et al. 2000; Basdogan et al. 2000; Cockburn et al. 2005; Tholey et al. 2005;
Hokayem et al. 2006; El Saddik 2007]

R. Ferrell and T. B. Sheridan, “Supervisory control of remote
manipulation,” IEEE Spectrum, vol. 4, no. 10, pp. 81–88, October 1967.



Haptic interaction in shared (virtual) environments

Subjective sense of togetherness in shared environments is significantly
improved when haptic feedback is provided [C. Basdogan et al., 2000]



Collaborative Haptics



Haptics
Kinesthetic Perception Tactile Perception

Image Source: Katsunari Sato,
Dept. of MEIP, The University of Tokyo/Japan

Position & Forces sense of touch of the skin

Percep'on
of

form,
posi'on,
surface
texture,
s'ﬀness,
fric'on,
temperature,
etc.



Haptic Communications

Position/Velocity

Internet

Force Feedback

1000 Hz sampling rate



Properties of haptic data streams

§  Data format:
§  Degrees of freedom: between 1 and >20
§  Sampling frequency: up to 1000Hz
§  Sampling resolution: up to 16bit

§  Transmission properties:
§  Very strict delay constraints (stability)
§  Control loop is closed by communication system
§  High packet rates (up to 1000 pkts/s)
Block-based coding not feasible !
§  Bad payload/header ratio



Outline

§  Early work in haptic data compression / reduction
§  Perceptual online data reduction of haptic signals
§  Extension to multi-DoF
§  Perceptual offline coding of haptic signals
§  Error-resilient haptic communication



Early work in haptic data compression / reduction

§  Lossy compression of haptic payload (different
sampling, quantization and entropy coding schemes)
§  Hikichi et al., ICME 2001
§  Shahabi et al., ICME 2002
§  Ortega and Liu, Prentice Hall 2002
§  Borst, WorldHaptics 2005

Packet rate reduction is not addressed !

§  Packet rate reduction
§  Otanez et al., American Control Conf., 2002.

Human haptic perception is not exploited !



Outline




Weber’s law

Just Noticeable Difference (JND)

ΔI
= constant
I
Stimulus Intensity
10g
10g

Ernst Heinrich Weber (1795-1878)
1kg 1kg+10g 100g 110g
Image Source: Max Planck Institute
for the History of Science, Berlin
http://vlp.mpiwg-berlin.mpg.de/people/data?id=per154



Perceptual haptic data reduction

§  Exploit limits of human haptic perception

§  Only transmit packets which cause a perceivable change

§  Based on Weber’s Law of Just Noticeable Differences (JND)

§  Examples of JND [Jones et al. 1992, Burdea 1996]
§  Arm position: 8%

§  Forces at a finger: 5-14%

§  Arm velocity: 8%

§  Moments at a finger: 13%



Perceptual deadband coding
ΔI
Sender: JND (Weber) = constant
I
A

t
Signal Updates

Receiver:

A

t
P. Hinterseer et al., IEEE Trans. on Signal Processing, 2008.


Predictive coding and filtering
Position + Velocity

LP Prediction
Prediction
Filter
Deadband
HSI TOP

Prediction LP
Prediction
Filter
Deadband

Force

§  Deadband applied to difference between true and predicted signal
§  Low-pass filtering of input signals
§  Noise reduction
§  Removal of not perceivable or not displayable frequencies

P. Hinterseer et al., IEEE Trans. on Signal Processing, 2008.


Combination with predictive coding

A A

t t
Predicted Signal Input Signal

Only samples that differ from the predicted signal by more than the
Weber JND have to be encoded



Results for 1 DoF experiment
1000
Velocity LP + LinPred
900 Force LP + LinPred
800 Velocity LinPred
Mean perception threshold Force LinPred
Packet rate [pkts/s]

700 of the test persons
600

500

400

300

200
≈ 90%
100

0 ≈ 94%
0 5 10 15 20 25 30 35 40
Deadband [%]


Alternative prediction approach: local object model

Environment
OP Model Network Model
HSI

TOP

DB

Define a local model of the geometric structure and
impedance properties of the currently touched surface.

è Haptic rendering based on local surface model



Local surface models



Example



Results
Rate-Quality Performance

X. Xu et al., IEEE HAVE 2011


Outline




Multi-DoF extension

§  Independent usage of 1-DoF deadband on
components of multi-DoF data?
§  Packet rate is determined by lowest magnitude
§  Low efficiency
§  Performance decreases with increasing number of degrees of
freedom

§  Alternative: multi-DoF deadband



Multi-DoF isotropic deadzone
y y

2-DoF:

x x

y y

3-DoF:

x x
z J. Drösler 2000
z

Discard haptic sample Encode haptic sample



Direction Adaptive Perceptual Force Deadzone

Based on the psychophysical findings on
Force Feedback Discrimination presented in

H. Tan et al., “Force-direction discrimination is
not influenced by reference force direction and
amplitude,” Haptics-e, 2006.

H. Pongrac et al., “Limitations of human 3d
force discrimination,” in Proc. Human-Centered
Robotics Systems 2006.

Direction-adaptive force deadzone
J. Kammerl et al., IEEE HAVE 2010



Outline




Haptic recording and replay

SensAble Technologies Source: DHZ/SFB 453
Position + Velocity

Recorder / Player
Force Feedback

Video

Operator with Teleoperator in
Human-System Inferface Compression remote environment

Data Storage



Haptic recording and replay

§  Challenge: Simultaneous display of force and position/
motion
§  Realization of playback
§  Position guidance [Crossan et al. 2006, El Saddik et al. 2007]
§  (Visual) substitution of competing haptic signals [Henmi et al.
1998, Williams 2004, Corno et al. 2006 ]
§  Application scenarios
§  posterior performance analysis / documentation
§  training and teaching
§  entertainment



Deadband-based offline compression of recorded haptic
signals
100 - no difference

75 – perceptable,
but not disturbing

50 – slighly disturbing

25 – disturbing

•  High transparency up to a deadband size of k=0.4
•  More than 95% of samples dropped
J. Kammerl and E. Steinbach, ACM Multimedia 2008



Outline




Error-resilient haptic data communication

Artifacts

Bouncing Roughness Glue Effect
F. Brandi, J. Kammerl, and E. Steinbach, ACM Multimedia 2010



Markov Decision Process (Binary Tree)

Markov
Channel
Decision
Predictor Model
Tree

ACKs
Perceptual DB
Position / Velocity
HSI TOP
Force Feedback
Perceptual DB
ACKs

Operator with Markov Predictor Teleoperator in
Channel
Human-System Interface Model
Decision remote Environment
Tree

F. Brandi, J. Kammerl, and E. Steinbach, ACM Multimedia 2010.



Summary

§  Perceptual haptic data reduction
§  Based on Weber’s law of just noticeable differences
§  1-dof, multi-dof
§  90-95% packet rate reduction
§  Similar performance for recording and replay
§  Error-resilient haptic data communication



Outlook: Selected open issues

§  How to integrate temporal aspects in human haptic
perception?
§  Haptic communication for area-based haptic sensing
and actuation (including tactile information)
§  Objective measures for immersiveness?
§  Perceptual coding of wave variables?
§  Joint data reduction and multiplexing for audio/video/
haptics?
§  …



Acknowledgments
§  Current and former PhD students: P. Hinterseer, J. Kammerl, F.
Brandi, R. Chaudhari, X. Xu, B. Cezmici, C. Schuwerk
§  Collaborators
§  S. Chaudhuri (IIT Bombay)
§  M. Buss, S. Hirche and I. Vittorias (Institute of Control Eng. @ TUM)
§  B. Färber and V. Nitsch (University of Armed Forces Munich)
§  A. El Saddik and J. Cha (University of Ottawa)
§  B. Hannaford and H. King (University of Washington)
§  Funding
§  DFG SFB 453: High-fidelity telepresence and teleaction
§  DFG STE 1093/4-1
§  ERC Grant 258941 “ProHaptics”
§  European-Brazilian Network for Academic Exchange EUBRANEX


Haptic Communications

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (19)

Similaire à Haptic Communications

Similaire à Haptic Communications (20)

Plus de Förderverein Technische Fakultät

Plus de Förderverein Technische Fakultät (20)

Dernier

Dernier (20)

Haptic Communications