The document discusses Receive Packet Steering (RPS) and Receive Flow Steering (RFS). It shows that enabling both RPS and RFS can increase packets per second (PPS) throughput by up to 35% compared to only using RPS or not using either. It analyzes the CPU load balancing performance with different configurations and explains how RPS and RFS work at the hardware and software levels to distribute network processing across CPU cores.

1. RPS/RFS
twitter
:
@gokzy

2. RPS/RFS
• NIC
• google
• kernel
2.6.35
2

3. No
RPS/RFS
3

4. RPS
Only
4

5. RPS/RFS
5

6. • iperf
– 4 /4
•
– UDP 64byte
6

7. -­‐
Server
• CPU
:
4Core
-­‐
Intel
Xeon
E5310
@
1.60GHz
(x2)
• MEM
:DDR3
Memory
2GB
• NIC
:
Broadcom
NetXtreme
BCM5751
(on
board)
• OS
:
Ubuntu
Server
10.10
• Kernel
:
2.6.36.1
7

8. &
• CPU
mask
:
ff
– /sys/class/net/eth[0-­‐9]/queue
/rx-­‐[0-­‐9]/rps_cpus
– CPU
• RPS
Flow
Conter
:
256
– /sys/class/net/eth[0-­‐9]/queue/rx-­‐[0-­‐9]/rps_flow_cnt
• Sock
Flow
Entries
:
256
– /proc/sys/net/core/rps_sock_flow_entries
8

9. -­‐
PPS
300000
250000
Packet
Per
Second
200000
150000
100000
50000
0
NO
RPS/RFS
RPS
ONLY
RFS
9

10. •
– CPU
• net_rx_action()
– CPU
• recvmsg() CPU
?
• iperf CPU4-­‐7
10

11. CPU
4000000
NO_RPS
3500000
3370072
RPS
ONLY
3371706
3000000
RFS
2500000
2000000
1679144
1494721
1486299
1500000
1336311
1177968
1500483
1355766
1000000
500000
0
0
10
0
1
13664
0
4
2
0
0
0
0
0
2
0
CPU0
CPU1
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
11

12. HW
4000000
HW CPU NO_RPS
3500000
RPS
ONLY
3000000
RFS
2500000
2000000
1500000
1000000
500000
0
CPU0
CPU1
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
12

13. 4000000
NO_RPS
3500000
RPS
ONLY
3000000
RFS
2500000
CPU
2000000
1500000
1000000
500000
0
CPU0
CPU1
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
13

14. • RPS/RFS UP
– PPS 35%
UP
• RPS CPU
– CPU
• RFS recvmsg
CPU
14

15.
15

16. • RPS/RFS
–
• CPU
– get_rps_cpu
• 2.6.37-­‐rc5
16

17. RPS/RFS
~ ~
struct net_device
:
struct net_dev_rx_queue *_rx struct netdev_rx_queue
: struct rps_map *rps_map
RPS struct rps_dev_flow_table *rps_flow_table
struct rps_map
unsigned int len
struct rps_dev_flow_table RFS
unsigned int mask
struct rcu_head rcu
struct rcu_head rcu
u16 cpus[0]
struct work_struct free_work
struct rps_dev_flow flows[0]
struct rps_dev_flow
u16 cpu
u16 fill
unsigned int last_qtail
17

18. RPS/RFS
~ ~
–
•
struct rps_sock_flow_table
unsigned int mask
u16 ents[0]
18

19. SW CPU
• get_rps_cpu [net/core/dev.c]
• CPU
•
– IP
– jhash_3words
– sk_buff rxhash
• RPS RFS CPU
19

20. RPS
• rps_map-­‐>cpus[] CPU
ID
– mask CPU
• map-­‐>cpus[
(
(u64)
skb-­‐>rxhash
*
map-­‐>len
)
>>
32]
– rxhash
:
– map-­‐>len
:
mask CPU
• 8Core
mask
0x69(0b01101001)
rps_map->len
rps_map->cpus 1 2 4 7 20

21. RFS
•
• 2 …
•
21

22. 2
• rps_sock_flow_table
– recvmsg() sendmsg()
CPU
– “desired
CPU”
• rps_dev_flow_table
–
CPU ?
–
– “current
CPU”
22

23. -­‐
1
• desired
CPU
,
current
CPU unset
– RPS (map-­‐>cpus )
• desired
CPU unset
or
offline
– current
CPU
• current
CPU
==
desired
CPU
–
23

24. -­‐
2
• current
CPU
!=
desired
CPU
–
– backlog
queue
counter
– rps_sock_flow_table CPU
backlog
queue
•
– process_backlog
24

25. netif_receive_skb
CPU
enqueue_to_backlog
skb_queue_len(&sd->input_pkt_queue)
== 0
backlog
queue
__napi_schedule skb_queue_tail()
enqueue
25

26. sock_flow_table
Ethernet
get_rps_cpu()
IP
-­‐
IP
( )
–
UDP
/
TCP
__udp_queue_rcv_skb()
( )
–
UDP
/
TCP
inet_recvmsg()
-­‐
HTTP
26

27. sock_flow_table
• inet_recvmsg
sock
sock_rps_record_flow(sk)
rps_sock_record_flow()
cpu
=
raw_smp_processorid()
sock_flow_table-­‐>ents[sk-­‐>sk_hash
&
sock_flow_table-­‐>mask]
=
cpu
• sk-­‐>sk_hash?
27

28. sk-­‐>sk_rxhash (ry
• __udp_queue_rec_skb()
– udp_rcv ( )
• skb-­‐
>rxhash
28

29. sock_flow_table
•
• sock
– hash sock-­‐>sk_hash
• recvmsg sock_flow_table[hash
&
mask] CPU ID
•
29

30. • 0xffff
– NO_RPS_CPU
=
0xffff
[
76.198633]
rxhash
hash
[
76.198640]
skb-­‐>rxhash
&
flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.198644]
tcpu
:
ffff
[
76.198647]
skb-­‐>rxhash
&
sock_flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.198650]
next_cpu
:
ffff
[
76.198655]
[if(map)]
tcpu
:
2
[
76.198657]
[cpu_online]
tcpu
:
2
30

31. 2 sock_flow_table
• flow_table
[
76.221073]
rxhash
hash
[
76.221078]
skb-­‐>rxhash
&
flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.221082]
tcpu
:
ffff
[
76.221085]
skb-­‐>rxhash
&
sock_flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.221088]
next_cpu
:
2
[
76.221090]
[cpu
chek
a]
-­‐1
[
76.221093]
[cpu
chek
b]
-­‐1
[
76.221095]
[got_hash]
cpu
:
2
31

32. 3 CPU
• flow_table CPU
[
76.232303]
rxhash
hash
[
76.232308]
skb-­‐>rxhash
&
flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.232312]
tcpu
:
2
[
76.232314]
skb-­‐>rxhash
&
sock_flow_table-­‐>mask
:
b6c7b0a3
&
f
=
3
[
76.232317]
next_cpu
:
2
[
76.232320]
cpu
:
2
32