This document discusses quality of service assessment for mobile speech services, including in vehicles. It contains the following key points: 1. The impact of terminal design on speech quality and methods for assessing quality are discussed, including standards for testing mobile terminals. 2. Driver distraction from speech services in cars is addressed, noting the importance of superior communication quality and seamless interaction to reduce distraction. 3. Ensuring high speech intelligibility and naturalness in vehicles is important to reduce driver distraction from poor sound quality. Wideband services and efficient use of vehicle audio systems can help achieve this.

1. Colombia, 23-24 September 2013
QoS assessment for mobile speech services
including user interfaces in vehicles and
hands-free communication
Dr.-Ing. H. W. Gierlich
Head of Telecom Division
CITEL (PCC.I)/ ITU Forum on Information and
Communication Technology Service:
Quality, Control and Surveillance
(Cartagena de Indias, Colombia, 23-24 September 2013)

2. Colombia, 23-24 September 2013 2
Outline
Introduction
The impact of terminal design on
speech quality
Assessment methods and standards
for mobile terminals
Driver distraction and speech
services in cars
Summary

3. Users Expectation (QoE)
Colombia, 23-24 September 2013 3
Speech
Quality
... ...
Sound Quality &
Naturalness
Intelligibility
Talking
Effort
Conversational
Effort
Double Talk
Performance
Expectation
Transmission of
Background Noise
Speech
Characteristics
Individual
Listening
Network
Conditions
Environmental
Conditions
Listening
Effort

4. Signal Processing in Modern Terminals
Colombia, 23-24 September 2013 4
Modulation
/
Demodulation
H(f )
-
NLP
CN
EC
Decoder
Coder
Loudspeaker
DecoderComfort
Noise
Mikrofon –
(Array)
EC
AGC/
companding
… Coder
Noise
cancellatio
n
+
NR
z.B. GSM/UMTS
SE
Speech-
“enhancement”speech recognition/
control
EQ
Important: All signal processing is
non-linear, time variant and
targeted to speech enhamcement

5. Speech Quality
Colombia, 23-24 September 2013 5
TESTING
…in general should be based on users perception
…may be based on simple signals and procedures modelling aspects
of users perception (e.g. loudness)
…may be based on complex signals and advanced procedures
modelling aspects of users perception (e. g. switching and double talk
tests)
…may be based on perceptual models using psychoacoustically
motivated procedures (models of the human hearing)
USERS PERCEPTION
…is subjective and generally different between subjects
…is based on experience
…is cultural dependant

6. Colombia, 23-24 September 2013 6
Outline
Introduction
The impact of terminal design on
speech quality
Assessment methods and standards
for mobile terminals
Driver distraction and speech
services in cars
Summary

7. Speech Quality Evaluation
Colombia, 23-24 September 2013 7
Speech-
Quality
Echo
Performance
Receiving
Double Talk
Quality of
Background
Noise Transm.
Sending

8. Terminal Testing
Bring the world to the lab:
Simulate user
Simulate user behaviour
Simulate user environments
Simulate network and access points
Simulate network conditions
Valdidate correct NTP access
Colombia, 23-24 September 2013 8

9. The Typical Lab Setup
Colombia, 23-24 September 2013 9
Sub-
woofer
Testsystem
Network
Simulation
Background
Noise
Simulation

10. Relevant Standards
GPP TS 51.010 (GSM 11.10)
3GPP TS 26.131 & TS 26.132
3GPP2 C.S0056
TIA 928 (CDMA/TDMA)
CTIA
GSMA HDvoice standard
For more advanced testing:
ETSI TS 103 737 – TS 103740 standard
series
Colombia, 23-24 September 2013 10

11. Moste Important Test Parameters
Enabling a Good QoS Performance
Basic acoustic tests e.g. Loudness Ratings
(ITU-T P.79), Frequency response tests…
Delay and echo tests
Speech quality tests – with and without
background noise
Conversational test including double talk,
switching…
Important: Test signals are mostly speech
based – ITU-T P.501
Colombia, 23-24 September 2013 11

12. The Impcat of Delay on Conversational
Dynamics
Colombia, 23-24 September 2013 12
Best case between fixed & mobile network (no satelite):
~150 ms
Worst case across networks:
>> 500 ms
For superior conversational quality: delay < ~150 ms
(from the users perspective)
Network
Terminal A Terminal B0 ms
(.wav)
150 ms
(.wav)500 ms
(.wav)

13. Conversational/Double Talk
Test Signal
Colombia, 23-24 September 2013 13
Word fully masked, one
speaker continuosly
talking
Word in partially masked
Word conicident with the other channel speaker,
Simulating delay and unwanted double talk
Double talk sentences
Receive
Send
Male Female Male Female
Short isolated words
Word in a speech pause

14. Summary of Quality Parameters Using
„Quality Pies“ according to ITU-T P.505
Colombia, 23-24 September 2013 14
TMOS SND
2
3
4
5
SLR
2/14
4/12
6/10
8
D-Value
-10
0
-5
5
BGNT (far)
10
0DT Type
2c
2b
2a
1
Subj. Exp. Assessment
poorfairgood
TCLw max
30
40
50
70
TCLw nom
30
40
50
70
Echo vs. time
not Ok
OK
TMOS RCV
2
3
1,5
4
RLR nom
8/-4
6/-2
4/0
2
G-MOS CAR
2
3,5
5
N-MOS CAR
2
3,5
5
S-MOS CAR
2
3,5
5
G-MOS Mensa
2
3,4
5
N-MOS Mensa
2
3,4
5
S-MOS Mensa
2
3,4
5
G-MOS ROAD
1,5
2,3
3,2
N-MOS ROAD
1,5
2,3
3,2
S-MOS ROAD
1,5
2,3
3,2
G-MOS TRAIN
2
3
4,3
N-MOS TRAIN
2
3
4,3
S-MOS TRAIN
2
3
4,3
Phone 4
Conv. Quality
TMOS SND
2
3
4
5
SLR
2/14
4/12
6/10
8
D-Value
-10
0
-5
5
BGNT (far)
10
0DT Type
2c
2b
2a
1
Subj. Exp. Assessment
poorfairgood
TCLw max
30
40
50
70
TCLw nom
30
40
50
70
Echo vs. time
not Ok
OK
TMOS RCV
2
3
1,5
4
RLR nom
8/-4
6/-2
4/0
2
G-MOS CAR
2
3,5
5
N-MOS CAR
2
3,5
5
S-MOS CAR
2
3,5
5
G-MOS Mensa
2
3,4
5
N-MOS Mensa
2
3,4
5
S-MOS Mensa
2
3,4
5
G-MOS ROAD
1,5
2,3
3,2
N-MOS ROAD
1,5
2,3
3,2
S-MOS ROAD
1,5
2,3
3,2
G-MOS TRAIN
2
3
4,3
N-MOS TRAIN
2
3
4,3
S-MOS TRAIN
2
3
4,3
Phone 3
Conv. Quality
TMOS SND
2
3
4
5
SLR
2/14
4/12
6/10
8
D-Value
-10
0
-5
5
BGNT (far)
10
0DT Type
2c
2b
2a
1
Subj. Exp. Assessment
poorfairgood
TCLw max
30
40
50
70
TCLw nom
30
40
50
70
Echo vs. time
not Ok
OK
TMOS RCV
2
3
1,5
4
RLR nom
8/-4
6/-2
4/0
2
G-MOS CAR
2
3,5
5
N-MOS CAR
2
3,5
5
S-MOS CAR
2
3,5
5
G-MOS Mensa
2
3,4
5
N-MOS Mensa
2
3,4
5
S-MOS Mensa
2
3,4
5
G-MOS ROAD
1,5
2,3
3,2
N-MOS ROAD
1,5
2,3
3,2
S-MOS ROAD
1,5
2,3
3,2
G-MOS TRAIN
2
3
4,3
N-MOS TRAIN
2
3
4,3
S-MOS TRAIN
2
3
4,3
Phone 2
Conv. Quality
TMOS SND
2
3
4
5
SLR
2/14
4/12
6/10
8
D-Value
-10
0
-5
5
BGNT (far)
10
0DT Type
2c
2b
2a
1
Subj. Exp. Assessment
poorfairgood
TCLw max
30
40
50
70
TCLw nom
30
40
50
70
Echo vs. time
not Ok
OK
TMOS RCV
2
3
1,5
4
RLR nom
8/-4
6/-2
4/0
2
G-MOS CAR
2
3,5
5
N-MOS CAR
2
3,5
5
S-MOS CAR
2
3,5
5
G-MOS Mensa
2
3,4
5
N-MOS Mensa
2
3,4
5
S-MOS Mensa
2
3,4
5
G-MOS ROAD
1,5
2,3
3,2
N-MOS ROAD
1,5
2,3
3,2
S-MOS ROAD
1,5
2,3
3,2
G-MOS TRAIN
2
3
4,3
N-MOS TRAIN
2
3
4,3
S-MOS TRAIN
2
3
4,3
Phone 1
Conv. Quality
Background Noise Performance

15. Colombia, 23-24 September 2013 15
Outline
Introduction
The impact of terminal design on
speech quality
Assessment methods and standards
for mobile terminals
Driver distraction and speech
services in cars
Summary

16. Driver Distraction
Colombia, 23-24 September 2013 16
Physical/manual distraction
Manual operation of devices
Visual distraction
Watching information other than road
Auditory distraction
Focusing on auditory events not related to
the driving task
Cognitive distraction
Occupied by non driving related tasks

17. Technology Based Distraction in Cars
Colombia, 23-24 September 2013 17
Car entertainment systems
Navigation systems
Phones for speech communication
Car information systems
Text messanging systems
„Texting while driving“
:

18. To be Avoided
Colombia, 23-24 September 2013 18
In general all activities distracting
from the driving task, esp.:
Visual cues which require different focus
than road
Non intuitive manual operation of the car
Loading the human auditory system by
distracting cues
Unnatural and low quality dialogs and
communications

19. Driving Task
Colombia, 23-24 September 2013 19
The driving task
=> Auditory Channel and Speech “available”
mostly occupied:
visual system
mainly involved:
hands and legs
involved:
auditory system
not involved:
talking

20. Speech Based Services in Cars
Colombia, 23-24 September 2013 20
o The main speech applications:
• Speech recognition systems
• Speech dialog systems
• Text to speech systems
• Speech enhancement for
communication systems
• Hands-free communication
• Enhanced in-car communication
systems between passengers

21. Communicational Quality –
Drivers Distraction
Colombia, 23-24 September 2013 21
Superior
Communication
Quality -> Reduced
Drivers Distraction
Seamless Interaction
-> Low Delay,
Perfekt Double Talk
Performance
Perfekt Intelligibility,
High Speech
Naturalness at the
Far End
Perfekt Intelligibility,
High Speech
Naturalness in the
Car

22. Hands-Free in Cars
Colombia, 23-24 September 2013 22
Why hands-free in
cars?
Reduce phsyical
distraction
(dialing, holding
the phone…)
Reduce visual
distraction
(watching display,
keyboard …)

23. Superior Sound Quality &
Intelligibility in the Car
Colombia, 23-24 September 2013 23
o Wideband services in mobile networks available soon
->
o Enabling wideband telephony (100 Hz- 8 kHz) in cars
• Fullband
• Narrow band (car)
• Wideband (car)
o Efficient use of the high quality audio systems in
cars:
• Getting superior sound quality
• Increasing speech intelligibility
• Increasing naturalness of a conversation
• Reduce drivers distraction due to poor speech sound quality

24. Colombia, 23-24 September 2013 24
Bluetooth Link
Phone
adapter &
antenna
amplifier
Radio Link
Hands-free
signal
processing
Car
audio
system

25. Car Hands-Free Testing
Colombia, 23-24 September 2013 25
analysisanalysis
Mobiles Sound Playback
System (can be installed in any
vehicle)
Subwoofer
HFT
ACQUA
Background
Noise
Generator
and Analyzer
MFE VI
USB
Frontend
PEQ IV
Programmable
Equalizer
PEQ IV
Programmable
Equalizer
PEQ IV
Programmable
Equalizer
Radio Tester
(Analog)
(Digital)
AES / EBU
(Digital)
AES / EBU
(Digital)
USB
(Digital)
AES / EBU
Subwoofer
HFT
Subwoofer
HFT
ACQUA
Background
Noise
Generator
and Analyzer
MFE VI
USB
Frontend
PEQ IV
Programmable
Equalizer
PEQ IV
Programmable
Equalizer
PEQ IV
Programmable
Equalizer
Radio Tester
(Analog)
(Digital)
AES / EBU
(Digital)
AES / EBU
(Digital)
USB
(Digital)
AES / EBU
Test System

26. ITU-T Standards Contributing to
Reduced Drivers Distraction
Colombia, 23-24 September 2013 26
ITU-T standards for Hands-Free Communication:
• ITU-T P.1100 for narrowband hands-free
• ITU-T P.1110 for wideband hands-free
ITU-T standards work in FG CarCOM:
• New work on subsystem requirements for Hands-Free
Systems in Cars
ITU-T standards for Dialog Systems:
• P.851: Subj. evaluation of dialog systems
• Suppl. 24 to P. Rec.: Parameters describing
the interaction with spoken dialog systems

27. The Main Section in
ITU-T P.1100 & P.1110
Colombia, 23-24 September 2013 27
Definition of the test arrangement in a car
Description of a digital interface concept for
development and debugging
Microphone test specification for separate
microphones in a car
Measurement parameters and requirements for
hands-free terminals
Bluetooth test interface for validation of
telephone performance
Subjective test strategy for car to car
communication

28. Conclusions and Recommendations
Mobile terminal testing is essential to provide
sufficient or even superior QoE to the user – many
operator have specific terminal standards for their
suppliers
A variety of parameters contribute to the overall
(conversational) speech quality
Network conditions have to be considered in the
testing
Low driver distraction is essential to offer mobile
services
Speech based operation is the key to the success of
mobile services in cars
Both, terminals and networks have to be
thoroughly engineered and adapted to each other
Colombia, 23-24 September 2013 28

29. Tandeming of Signal Processing in
Terminals & Networks
Colombia, 23-24 September 2013 29
Base
Station
EC
ES
NR
Mobile
Phone
EC
ES
NR
gain gain
Mobile
Network
Netw. 1
Netw. 2