口試投影片(詹智傑) Final

自適性串流架構下之視訊影像
品質最佳化設計與實作
指導教授：楊士萱
學生：詹智傑
Design and Implementation of Quality Optimization
Strategy in Adaptive Streaming

Outline
• Introduction to MPEG-DASH
• Related Works
• Proposed State-based Strategy
• Results and Future Works
2

Outline
• Introduction to MPEG-DASH
• Related Works
3

Introduction to MPEG-DASH
• Dynamic Adaptive Streaming over HTTP
– 由3GPP與MPEG一起訂定的自適性串流標準
– 屬於HTTP/TCP的串流協定
– 唯一的自適性串流標準
4

– 何謂自適性串流(Adaptive bit-rate streaming)
• 依據環境與應用需求即時性的調整視訊串流之碼率
– 視訊之解析度(Resolution)與品質(Quality)需求
– 網路狀態與撥放延遲(playback delay)
– 客戶端之暫存器使用效率(Video buffer)
– 使用終端之計算能力與複雜度
• 目的是獲得更佳之使用經驗品質(Quality of Experience,
QoE)與頻寬使用效率(Bandwidth efficiency)
5

– 使用HTTP/TCP (not RTSP/UDP)的優勢
• 多數使用終端支援HTTP 協定
• TCP能提供高可靠性(reliable)且有秩序的傳送(ordered
transmission) 並且能避免NAT與防火牆之阻擋問題
• HTTP-based streaming能夠重複使用已建置之 HTTP
servers與caches做資料內容(content delivery)的傳遞
• HTTP-based streaming將串流連線之控制權由Server端
移轉至Client端
6

– 世界上第一個也是唯一的International standard
• Apple HTTP Live Streaming (Apple-HLS)
• Microsoft Live Smooth Streaming (Microsoft-LSS)
• Adobe Dynamic Streaming for Flash (ADOBE-DSF)
• OPEN IPTV Forum: HTTP Adaptive Streaming
• Dynamic Adaptive Streaming over HTTP (MPEG-DASH)
• …
7
Draft International Standard : 2011 Jan
International Standard : 2011 Nov
ISO/IEC International Standard : 2012 April
ISO/IEC Technical Corrigendum 1 : 2013 May

• 系統架構(System Architecture)
8
HTTP server DASH client
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Media
Presentation
Description
(MPD)
Control heuristics
HTTP client
MPD parser
Segment
parser
Media
player
HTTP 1.1
視訊串流服務
之”目錄”資料模型
視訊串流服務
之”媒體”資料模型

• 資料模型(Data Model)
– Media Presentation Description (MPD)
• Descript the video segment’s information in xml
ex: timing, codec, resolutions, bit-rate … etc.
– Segments
• The media data.
• ex: mp4, m4s, mp3 … etc.
9

• 資料模型(Data Model)
– Media Presentation Description (MPD)
10
Media presentation
Description
Period ID = 1,
start=0 s
Period ID = 2,
start=100 s
Period ID = 3,
start=200 s
Period ID = 2
start=100s
Adaptation
Set 0
Adaptation
Set 1
Adaptation
Set 2
Adaptation set 1
Representation 1
5Mbps
Representation 2
2Mbps
Representation 3
500kbps
Representation 2
2 Mbytes
Segments

• 系統流程圖(System Flowchart)
11
Process MPD
HTTP Get MPD
Measure
Download Rate
Select Representation
Measure
Download Rate
HTTP Get Segment #1
HTTP Get Segment #2
#1
#2

• 客戶端系統流程圖(Client side system flowchart)
12
Main part of MPEG-DASH
Segment buffer

Major contribution of this thesis
• 實際建置一個包含伺服器與客戶端之視訊串
流系統，並加入MPEG-DASH框架
• 我們採用emulation的形式，相較於simulation
能提供更真實、更有修改與加入環境參數彈
性的系統
• 我們探索視覺品質(visual quality)、網路頻寬
(network bandwidth)以及暫存器使用量(buffer
usage)的關聯性，並提出一套自適性品質控
制以提升感知上的品質(perceived quality)
13

Outline
• Introduction of MPEG-DASH
• Related Works
– VLC Player MPEG-DASH rate-based Quality
Strategy
– Related Researches in MPEG-DASH
14

VLC Player Rate-based Quality Strategy
• 實作於VLC Player DASH
plug-in中
• Main consider principle
– 當暫存器容量低於預設
threshold，接收最低碼
率串流以加速暫存器容
量補充
15
< minimum
buffer threshold
> minimum
buffer threshold
NoneAt least one
Algorithm start
Initial
MPD Manager
Current Period
Get current Bandwidth
Check current
Buffer capacity
Set current bandwidth
as zero
Gather Representations with
same resolution of the player
Check the quantity
of Representations
in candidate set
Find target Representation
from every AdaptationSets
Find target Representation
from candidate set
Algorithm end
Stage one
Stage two

• Advantage
– 視訊片段緩衝器補充速度快
• Disadvantage
– 串流初始狀態的品質等級低
– 易受到網路瓶頸而品質崩潰
16

• Advantage
• Disadvantage
– 串流初始狀態的品質等級低
– 易受到網路瓶頸而品質崩潰
17
Bottleneck

Outline
• Related Works
– VLC Player MPEG-DASH rate-based Quality
Strategy
– Related Researches in MPEG-DASH
18

Related Researches in MPEG-DASH
• [23]以馬可夫鍊的方式對無線網路的頻寬變
化特性做分析與歸類，並提出找尋最佳視
訊品質的演算法。
• [25]探討如何使用可調式編碼(Scalable video
coding)提升網路儲存空間的使用效率。
• [26]以代理伺服器(Proxy)的角度出發，以網
路管理的QoE機制來替客戶選擇最佳的視訊
品質。
19

Related Researches in MPEG-DASH
• 雖然過去兩年MPEG-DASH相關的投稿文獻
絡繹不絕，但多數的共通點都如[23][25]採
用純模擬(Simulation)或數學推導，無法於實
際網路中驗證。
• 故本論文除了提出一套MPEG-DASH的品質
調控機制，也將實作一套系統進行驗證。
20

Outline
• Related Works
21

Proposed State-based Quality Strategy
• 本論文提出之DASH品質調控機制分 two
stage
– First Stage：以下載MPD的網路資訊預測前幾個
Segment的最佳品質
– Second Stage：依照網路與視訊片段暫存器使用
率來預測下一個Segment的最佳品質
22

• Flowchart of first stage
– Case 1:
• MPD檔中的第一個
Segment。
– Case 2:
• 新Adaptation set的第一
個Segment的決策流程。
– Case 3:
• 一般之Segment的決策流
程
23
Choise segment
Start
!First segment
!First segment
in MPD
No
1 2 3
No
Yes
Yes
Initial
Get target RepresentationList
Sort Representations
RepresentationStream

24
1
Download Initialization
Source URL
Download first five segment
by using MPD’s bitrates as ref.
Compute and save five
segment’s downloaded bitrates
End
2
Download Initialization
Source URL
Choice target level according to
pre-algorithm’s last five segments
Compute and save
End
Download target level segments
3
Choice target level according to
five pre-segments
Download target level segments
Compute and save
End
Case 1: Case 2: Case 3:

• Finite state machine of second stage
25
State 1-1
State 2-3
State 2-2
State 2-1
Network Throughputs
> Highest Representation
Network Throughputs
< Highest Representation
Network Throughputs
< 80% Representations
Network Throughputs
< 80% Representations
Network
Throughputs
(Bps)
Segment
Buffer
(%)0
Adaptation Set’s
Representation
(Level)
Level n
Level 1
Level 2
State 3-2 State 3-1
Level k
Network Throughputs
< Representations Level 2
Network Throughputs
> Representations Level 2
Buffer
Fill persentage > 30%
Buffer
30% 50%
Buffer
Buffer
Case One
Case Two
Case Three

Outline
• Related Works
– Emulation Settings
– Objective Evaluation
– Subjective Evaluation
26

Emulation Settings
• 實驗前提
– 本論文探討“頻寬高於MPD所能提供的最低碼率
視訊片段”的情境
– 本論文實作的是On demand的視訊串流服務
– 本論文探討“Client有足夠計算能力解析MPD中
任一等級的串流”
– 本論文假設碼率與PSNR的關係為已知，故不須
於解碼的同時檢測畫面品質。
27

Emulation Settings
• 測試平台
28
Client Server
Name Asus PC Asus PC
CPU Intel Core 2 Quad
2.67GHz
Intel Core 2 Quad
2.67GHz
Memory 4G DDRII 2G DDRII
OS Windows 7 Ubuntu 12.04
Network Ethernet Ethernet

Emulation Settings
• 測試平台
29
HTTP Server
Apache Server
(on demand)
Network
Emulator
Client
PC
Notebook
HTTP
Internet
DASH-AVC/264
Encoder

Emulation Settings
• 所使用之開源式軟體
30
Open Source Software
Operating System Linux Ubuntu 12.04 LTE
Web HTTP Server Apache 2
Server Database MySQL
Language Perl, Bash script
Network Emulator Linux tc commands
Network Monitor libpcap & winpcap
Server Monitor Monitorix
Client side libraries libdash, libcurl, libxml2, libav

Emulation Settings
• 測試序列
31
Parameter Value
Length (min:sec) 09:46
Segment size (second/segment) 2
Numbers of Frames 14315
Video codec H.264/AVC
Container MP4 & M4S
Frame rate (fps) 25 fps
GOP size (frame) 25
Resolution (pixel) 480 x 360

Emulation Settings
• 測試序列
32
Level Video bitrate Level Video bitrate
0 100 kbits/s 7 1300 kbits/s

Emulation Settings
• 序列特性：品質等級與碼率關係
33
The lower the bit-rate is,
the smaller the gap is.

Emulation Settings
• 網路模型
– 高品質穩態網路模型
• 網路頻寬維持於MPD最高品質等級之上
– 遞增型網路模型
• 網路頻寬由低頻寬以穩定之速率上升
– 遞減型網路模型
• 網路頻寬由高頻寬以穩定之速率下降
34

Emulation Settings
• 網路模型
35
0
1000
2000
3000
4000
5000
6000
7000
0 100 200 300 400
Bitrate(kbits/sec)
Time(s)
最高品質穩態網路模型遞增網路模型遞減網路模型

Outline
• Related Works
36

Objective Evaluation
• 頻寬使用效率分析
– 最高品質穩態網路模型
0
1000
2000
3000
4000
5000
6000
7000
0 10 20 30 40 50 60
Bitrate(kbit/sec)
Time(sec)
Bandwidth Throughput(VLC[31]) Throughput(Proposed)
ChosenBitrate(VLC[31]) ChosenBitrate(Proposed)
37

0
1000
2000
3000
4000
5000
6000
7000
0 10 20 30 40 50 60
Bitrate(kbit/sec)
Time(sec)
ChosenBitrate(VLC[31]) ChosenBitrate(Proposed)
38
0
1
2
3
4
5
6
7
8
9
10
11
12
13
0
1000
2000
3000
4000
5000
6000
7000
0 10 20 30 40 50 60
Level(#)
Bitrate(kbit/sec)
Time(sec)
ChosenLevel(VLC[31]) ChosenLevel(Proposed)

– 遞增網路模型
39

– 遞減網路模型
40

41

• 使用者觀賞品質分析
42

43
+4.02845 dB

• Average PSNR
44
PSNRY PSNRU PSNRV
Proposed 41.9227dB 47.8613 dB 49.9766 dB
VLC[31] 41.4475 dB 47.4507 dB 49.5599 dB
△PSNR +0.4752 dB +0.4106 dB +0.4167 dB

– 遞增網路模型
45

46

47

• Average PSNR
48
PSNRY PSNRU PSNRV
Proposed 39.311 dB 47.814 dB 48.930 dB
VLC[31] 38.728 dB 47.325 dB 48.449 dB
△PSNR +0.583 dB +0.489 dB +0.481dB

• Average PSNR (from 8000~12000 frame)
49
PSNRY PSNRU PSNRV
Proposed 38.411 dB 44.065 dB 44.54 dB
VLC[31] 36.93 dB 42.875 dB 43.325 dB
△PSNR +1.481 dB +1.19 dB +1.215 dB
畫面數: 8000~12000張

Outline
• Related Works
50

Subjective Evaluation
• 串流起始狀態
– 最高品質穩態網路模型 (frame 829)
Proposed (36.160dB) VLC[31] (27.810dB)
51

– 遞增網路模型 (frame 279)
52

– 遞減網路模型 (frame 373)
53

• 串流起始狀態(Demo)
Proposed VLC[31]
54

• 串流起始狀態(Demo)
Proposed VLC[31]
55

• 遞減網路模型低視訊品質狀態
– frame 9872
56

• 遞減網路模型低視訊品質狀態
– Demo
Proposed VLC[31]
57

Results and Future Works
• 本論文實作一套包含伺服器與客戶端的自
適性串流系統。
• 本論文在客戶端提出一套視訊品質評估機
制。
• 整體PSNR相較於[31]提升約0.5dB；在初始
狀態上，也能達到約1dB到4dB的提升；而
在網路瓶頸時也有1dB的提升。
58

Results and Future Works
• 提升品質調控機制的準確度
• 擴充其他視訊編碼標準與其相關特性探討
• 切割自適性串流機制與實際解碼的框架，
以做為智慧型家庭、微型視訊共享機制的
系統雛形。
59

Reference
• [1] Cisco, Cisco Visual Networking Index: Global Mobile Data Traffic
Forecast Update, 2012–2017, Feb 6, 2013.
• [2] R. Pantos, and W. May, HTTP Live Streaming, version 10, IETF Internet-
Draft, Oct 15, 2012.
• [3] Information technology — Dynamic Adaptive Streaming over HTTP
(DASH), ISO/IEC 23009-1:2012 ,Aprl 01, 2012.
• [4] Jack Y. B. Lee , Scalable Continuous Media Streaming Systems-
Architecture, Desgn, Analysis and Implementation, John Wiley & Sons Ltd,
June, 2005.
• [5] H.264: Advanced Video Coding for Generic Audiovisual Services, ITU-T
H.264, Jan, 2012.
• [6] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: Video
Coding for Next Generation Multimedia, John Wiley & Sons, 2003
• [7] 經濟部, 2003 數位內容產業年鑑趨勢篇第二章數位內容技術趨勢
60

Reference
• [8] Microsoft Corporation, IIS Smooth Streaming Technical Overview, Mar
25, 2009.
• [9] Apple Corporation, HTTP Live Streaming Overview, April 1, 2011.
(DASH) — Part 1: Media Presentation Description and Segment Formats,
AMENDMENT 1: Support for Event Messages and extended Audio Channel
Configuration, ISO/IEC 23009-1:2012/DAM 1, Dec 07, 2012.
(DASH) — Part 1: Media Presentation Description and Segment Formats,
TECHNICAL CORRIGENDUM 1, ISO/IEC 23009-1:2013/Cor.1:2012(E), Feb 02,
2013.
• [12] 蔡德明, 鳥哥的Linux私房菜基礎學習篇第二版, 上奇科技出版事務
處, March, 2006.
• [13] 蔡德明, 鳥哥的Linux私房菜伺服器架設篇第二版, 上奇科技出版事
務處, March, 2007.
61

Reference
• [14] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W. Weiss, An Archi-
tecture for Differentiated Services, IETF RFC 2475, Dec
1998, http://tools.ietf.org/html/rfc2475
• [15] M. Allman, V. Paxson, W. Stevens, TCP Congestion Control, IETF RFC
2581, Aprl 1999, http://tools.ietf.org/html/rfc2581
• [16] D. Robinson, K. Coar, The Common Gateway Interface (CGI) Version
1.1, IETF RFC 3875, Oct 2004, http://tools.ietf.org/html/rfc3875
• [17] Information technology — Dynamic adaptive streaming over HTTP
(DASH) — Part 2: Conformance and reference software, ISO/IEC 23009-
2:2012 ,May 28, 2012.
• [18] N. Cranley, P. Perry, and L. Murphy, “User perception of adapting
video quality,” ACM International Journal of Human-Computer Studies, pp.
637-647,Aug. 2006
62

Reference
• [19] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, "Overview of
the H.264/AVC video coding standard,” IEEE Transactions on Circuits and
Systems for Video Technology, vol.13, no.7, pp.560,576, July 2003
• [20] M. Wien, H. Schwarz, T. Oelbaum, "Performance Analysis of SVC,” IEEE
Transactions on Circuits and Systems for Video
Technology, vol.17, no.9, pp.1194,1203, Sept. 2007
• [21] A. Begen, T. Akgul, M. Baugher, " Watching Video over the Web: Part
1: Streaming Protocols,” IEEE Internet Computing, vol.15, no.3, pp.59-
63, May-June 2011
• [22] I. Sodagar, "The MPEG-DASH Standard for Multimedia Streaming Over
the Internet,” IEEE Multimedia, vol.18, no.4, pp.62-67, April 2011
• [23] S. Xiang, L. Cai, and J. Pan, “Adaptive scalable video streaming in
wireless networks,” MMSys'12 Proceedings of the 3rd Multimedia Systems
Conference, pp. 167-172, 2012
63

Reference
• [24] V. Swaminathan,and Wei Sheng, "Low latency live video streaming
using HTTP chunked encoding,” 2011 IEEE 13th International Workshop
on Multimedia Signal Processing (MMSP), pp.1,6, 17-19, Oct. 2011
• [25] Y. Sánchez, T. Schierl, C. Hellge, and T. Wiegand, “iDASH: improved
dynamic adaptive streaming over HTTP using scalable video coding,”
MMSys'11 Proceedings of the second annual ACM conference on
Multimedia systems, pp.257-264, Feb 2011
• [26] R.K.P. Mok, X. Luo, E. W. W. Chan, and R. K. C. Chang, "QDASH: A QoE-
aware DASH system,” MMSys'12 Proceedings of the 3rd Multimedia
Systems Conference, pp.11-22, Feb. 2012
• [27] O. Nemcic, M. Vranjes, and S. Rimac-Drlje, “Comparison of H.264/AVC
and MPEG-4 part 2 coded video,” Proc. Electronics in Marine
(ELMAR), pp.41,44, 12-14, Sept. 2007
64

Reference
• [28] T. Lohmar, T. Einarsson, P. Fröjdh,F. Gabin, M. Kampmann, “Dynamic
Adaptive HTTP Streaming of Live Content,” IEEE International Symposium
on a World of Wireless, Mobile and Multimedia Networks, June 20-
24, 2011
• [29] T. Stockhammer, “Dynamic Adaptive Streaming over HTTP–Standards
and Design Principles,” MMSys'11 Proceedings of the second annual ACM
conference on Multimedia systems, Feb 2011,
• [30] S. Lederer, C. Müller, and C. Timmerer, “Dynamic Adaptive Streaming
over HTTP Dataset,” In Proceedings of the ACM Multimedia Systems
Conference 2012, Chapel Hill, North Carolina, February 22-24, 2012.
• [31] C. Müller and C. Timmerer. “A VLC media player plugin enabling
dynamic adaptive streaming over HTTP,” In Proceedings of the 19th ACM
international conference on Multimedia, New York, NY, USA, 723-
726, 2011.
65

Reference
• [32] L. R., Romero, “A Dynamic Adaptive HTTP Streaming Video Service for
Google Android,” Oct 06, 2011, Available from:
http://dat.etsit.upm.es/~luciano /en/master-thesis.pdf
• [33] Jan Ozer, “What is streaming media protocol?,” http://www.streaming
media.com/Articles/Editorial/What-Is-.../What-Is-a-Streaming-Media-
Protocol-84496.aspx
• [34] Liz Gannes, “The Next Big Thing in Video: Adaptive Bitrate Streaming,”
http://pro. gigaom.com/blog/how-to-deliver-as-much-video-as-users-can-
take/
• [35] wiki Adaptive streaming:
http://en.wikipedia.org/wiki/Adaptive_bitrate_stre aming
• [36] Adaptive Streaming Comparison:
http://learn.iis.net/page.aspx/792/adaptive -streaming-comparison/
• [37] Video LAN Official page: http://www.videolan.org/vlc/index.html
66

Reference
• [38] September 2012 Web Server Survey :
http://news.netcraft.com/archives/2012/09/10/september-2012-web-
server-survey.html
• [39] Ubuntu documentation: Guide line for installing LAMP in Linux
Ubuntu : ht tps://help.ubuntu.com/community/ApacheMySQLPHP
• [40] Monitorix Project by Jordi Sanfeliu : http://www.monitorix.org/
• [41] RRDtool – About RRDtool : http://oss.oetiker.ch/rrdtool/index.en.html
• [42] libdash | bitmovin.net multimedia :
http://www.bitmovin.net/libdash/
• [43] libav－Open source audio and video processing tools :
http://libav.org/
• [44] Hierarchical token bucket theory :
http://luxik.cdi.cz/~devik/qos/htb/man ual/theory.html
• [45] Traffic control HOWTO : http://linux-ip.net/articles/Traffic-Control-
HOWTO/ 67

口試投影片(詹智傑) Final

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à 口試投影片(詹智傑) Final

Similaire à 口試投影片(詹智傑) Final (20)

Dernier

Dernier (20)

口試投影片(詹智傑) Final