Backplane Technology Overview for AdvancedTCA

Backplane Technology Overview
B k l T h l O i
for AdvancedTCA

Hui Chen Theresienstr. 90
Supervisor: Rainer Ohlendorf 80290 Munich
Germany

July, 13th 2007 www.lis.ei.tum.de

© Institute for
Integrated Systems

Outline

Introduction to AdvancedTCA
Interconnect Protocols
I t tP t l
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

Comparison of Interconnect Protocols
① Feature
② Protocol Efficiency
③ Effective Bandwidth
④ System Level Consideration

Conclusion
C l i

Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for
2
Hui Chen, July 13th 2007 Integrated Systems

What is AdvancedTCA ?
• AdvancedTCA (Advanced Telecom Computing Architecture)
Standard chassis/ backplane solution for high-availability
communications equipment
Adopted in 2003

• AdvancedTCA Chassis

3

Base Spec of AdvancedTCA
• Board dimension
1.2 inches in width
280 mm deep, ~322 mm high

• Power supply & Cooling
-48 V power feeds
200 Watts per board
Board of AdvancedTCA

• Backplane architecture
Zone 1: P
Z 1 Power, MManagement t
Zone 2: Data Transport Interface
Zone 3: User-defined I/O

Backplane of AdvancedTCA
4

Data Transport Interface
• Base Interface: Dual Star
Supports only Ethernet

• Fabric Interface: Dual Star/ Full Mesh
Supports various protocols
Max. 4 channels

(a) Dual Star configuration
• Dual Star
Redundant switch: eliminates single point of
failure
Effective for large redundant systems

• F ll M h
Full Mesh
+) Higher data throughput
-) Higher pins count & cost
Effective for
Eff ti f small systems
ll t
(b) Full Mesh configuration

5

Outline

① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
6

Protocol ① ― Ethernet
• Application:
Initial in Local Area Network
Successful in backplane
but defective in congestion management

• Congestion management
PAUSE function: all data flows are paused when only
one flow is congested
→ congestion spreading, when complex data p
g p g, p paths

• Topology
P2P connections: Ethernet Switch ↔ Endpoints Ethernet Topology

• Frame

Header: large

7

Protocol ② ― InfiniBand ⑴
• Application Processor Node

Initial in System Area Network
now also i b k l
l in backplane, chassis-to-chassis
h i t h i
Supercomputer, Linux cluster

• Topology
InfiniBand Switches + Channel Adapters
HCA: Host Channel Adapter in processor node
TCA: Target Channel Adapter in I/O node
I/O Node
HCA talks TCAs to enable remote
InfiniBand Topology
storage and connectivity

• Packet format

Larger payload size than Ethernet

8

Protocol ② ― InfiniBand ⑵
• Three link widths x1

Links x1 x4 x12 x4
Data Rate (Gb/s) 2 8 24
x12

• Supports Quality of Service (QoS)
QoS: a method to prioritize network traffic
Ensures most important data gets thru the network as quickly as possible
Defines traffic to many streams, streams to differentiated classes;
reserves bandwidth to streams and classes

9

Protocol ② ― InfiniBand ⑶
• Supports Quality of Service
Thru 16 Virtual Lanes (VLs)
Multiplex independent data streams
→ the same physical link
Represent a set of send & receive
p
buffers in a port
Each port: one management VL (VL15)
& up to 15 data VLs
Priorities

VL15 (highest)
VL14
…

VL0 Virtual Lanes

10

Protocol ③ ― StarFabric ⑴
• Application
Backplane, chassis-to-chassis
Embedded distributed processing
(e.g. multiprocessor systems)

• Topology
StarFabric Switch: six-port
Connects other switches/ bridges
Cascadable
StarFabric Bridge StarFabric Topology

Translates other protocols (e.g. PCI) into
StarFabric traffic
St F b i t ffi

• Not scalable
Operates only at 2 Gb/s
O t l t Gb/

11

Protocol ③ ― StarFabric ⑵
• Packet format

7 classes of traffic
asynchronous traffic (data)
multicast
…
isochronous traffic (voice & video)

• Backwards compatible to PCI
Converting a serial fabric to parallel PCI

12

Protocol ④ ― PCI Express ⑴
• Successor to PCI
Backplane, chip-to-chip interconnect
PC (e.g. Graphics cards)

• Topology
Switches
Multiple Endpoints: I/O devices
Host Bridge
To create an I/O hierarchy
PCI Express Topology

• Six link widths:
Links x1 x2 x4 x8 x16 x32
Data Rate (Gb/s) 2 4 8 16 32 64

13

Protocol ④ ― PCI Express ⑵
• Packet format

Thru
Thr 8 Traffic Classes (TCs) & 8 Virt al Channels (VCs)
Virtual
Each port : up to 8 VCs. VC0 (lowest priority), VC7 (highest)
TCs: differentiated services
Each
E h packet a TC t determine which VC b ff t d
k t TC: to d t i hi h buffer to drop packet i t
k t into
TC/VC mapping: ☺ TC[0:6]→VC0 TC7→VC[0:1]

An Example of TC/VC mapping
14

Protocol ⑤ ― RapidIO ⑴
• Application
Backplane, chip-to-chip interconnect
Embedded systems (e.g. DSPs)
Networking (e.g. Carrier Cards)

•T
Topology
l
A Switch, multiple Endpoints
Large systems: one / more host processors
For system exploration, initialization
exploration
Monitors system-level activity
RapidIO Topology

• Two link widths (x1 x4)
(x1,
Links x1 x4

Data Rate (Gb/s) 1 2 2.5 4 8 10

15

Protocol ⑤ ― RapidIO ⑵
• Packet format

Header: smaller than Ethernet:
1-byte source address ~ 28 Endpoints → sufficient for backplane applications
Payload size: relatively small

Six logical flows: as prioritized classes

16

Outline
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
17

Comparison – Feature

Feature GigE InfiniBand StarFabric PCI Expr
Expr. RapidIO

Transport PCB, Copper & PCB, Copper & PCB & PCB, Copper &
Media Fiber Fiber Optical
PCB only
CAT5 cable
Scalable Link
None Yes None Yes Yes
Widths

Encapsulated, Yes, Yes, Yes,
PCI Migration None requires SW
q transparent
p transparent
p requires SW

Max Signal
1 Gbps 2 Gbps 2 Gbps 2 Gbps 2.5 Gbps
Channel Rate

y
Max Payload
1500 B tes
Bytes 4096 B t
Bytes 128 B t
Bytes 4096 B t
Bytes 256 B tes
Bytes
Size

18

Comparison – Protocol Efficiency
• Protocol Efficiency = Payload / Packet size
• Payload < ~500B : RapidIO, highest efficiency smallest overhead
Payload > ~500B : PCI Express, highest
Express

19

Comparison – Effective Bandwidth
• Effective Bandwidth = Protocol efficiency × Bandwidth
• InfiniBand x4; PCIe x4; RapidIO 4x 2Gbps
• O
Overall: Gi E l
ll GigE, lowest
t

20

Comparison – System Level
• Quality of Service (QoS)

GigE
Gi E InfiniBand
I fi iB d StarFabric
St F b i PCI E
Expr. RapidIO
R idIO

Prioritized Traffic
－ 16 7 8 6
Classes

•P f
Performance
GigE InfiniBand StarFabric PCI Expr. RapidIO

Max Bandwidth for
M B d idth f
1 Gbps 8 Gbps 2 Gbps 8 Gbps 10 Gbps
AdvancedTCA, x4

Latency High Medium Medium Low Low
(~100 us) (1.29-2.6 us) (< 3 us) (0.3-0.7 us) (< 100 ns)

21

Outline
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
22

Conclusion
• Low bandwidth backplane applications
Ethernet +) well understood, low risk
-) high overhead, hi h l
) hi h h d high latency
-) not scalable

• High bandwidth (tradeoffs of flexibility vs. latency, overhead)
g ( y y, )
InfiniBand and StarFabric
Complex header: +) for complex routing
-) reduces effective bandwidth
RapidIO
Smaller packets: +) to lower latency
-) overhead: a higher percentage of the packet
-) reduces effective b d idth
) d ff ti bandwidth
PCI Express
Middle: +) design goal: software transparency
-) constrains the protocol raises latency
protocol,

23

Thank you for your attention !

Any questions ?

Special thanks to:
Rainer Ohlendorf

24

Backplane Technology Overview for AdvancedTCA

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (6)

Similaire à Backplane Technology Overview for AdvancedTCA

Similaire à Backplane Technology Overview for AdvancedTCA (20)

Dernier

Dernier (20)

Backplane Technology Overview for AdvancedTCA