IPv6 Development in ITB 2013 - development of IPv6 in the past, present and future - Presented at 100NGN Workshop in Jakarta 18 June 2013

1. What’s status:
IPv6 Implementation in ITB
Affan Basalamah
100NGN Workshop

2. # whoami
• Affan Basalamah
• IT Infra Manager
• Unit Sumber Daya
Informasi ITB
• affan@itb.ac.id
• @affanzbasalamah

3. Outline
The past
10 years
The
present
The
future

4. THE PAST 10 YEARS

5. Background
• ITB already implement IPv6 since 2001
– From SOI-ASIA program (www.soi.asia)
– Allocated subnet: 2001:d30:3::/48
• Registering IPv6 allocation to APNIC in 2007
– Allocated subnet: 2403:8000::/32

6. IPV6 NETWORK ADDRESS &
ROUTING

7. IPv6 External Connection
IPv6 Tunnel
• IPv6 address-family to all eBGP peers
HE.net
BGP Native

8. IPv6 Address Allocation
Distribution
• 2001:d30:3::/48 for NOC
• 2043:8000::/32 for ITB
– 1 buah /35 for ITB campus
– 7 buah /35 for future allocation

9. IPv6 Routing Implementation (1)
• FreeBSD/Linux-based PC Router (pre-2010)
– Quagga Routing Suite
– RIPng  OSPFv3
– BGP
• Dedicated Core Router/Switches (2010)
– Cisco Catalyst 6500 on Sup720-3B/Sup32
– Juniper SRX650

10. IPv6 Routing Implementation (2)
• PC Router with IPv4-only Layer 3 Switch
IPv6 Gateway
PC Router
IPv6
PC Router-1
IPv4-only
Layer 3 Switch
IPv6
Network

11. IPv6 Routing Implementation (3)
• PC Router with VLAN interface
• OSPFv3
IPv4-only
Layer 3 Switch
IPv6
PC Router-1
Tunnel & VLAN
IPv6
PC Router-2
IPv4-only
Layer 3 Switch
IPv4-only
Layer 3 Switch
IPv6
PC Router-3

12. IPv6 Routing Implementation (4)
• 802.1Q Trunk VLAN to distribute IPv6
subnet with Router Advertisement (RA)
IPv4-only
Layer 3 Switch
IPv4-only
Layer 3 Switch
PC
Layer 2 Switch
IPv6
PC Router
Layer 2 Switch
PC
Layer 2 Switch
PC
802.1Q Trunk

13. Campus Network

14. IPV6 APPLICATION

15. Aplikasi IPv6 di ITB
•
•
•
•
•
•
Operating System
DNS
WWW & FTP Server
Mail Exchange Server
Web Cache Proxy
Unicast & Multicast Stream

16. Operating System for Server
•
•
•
•
FreeBSD 9.x, 8.x, 7.x
CentOS Linux 6.x dan 5.x
OpenSolaris 2009.x
Windows Server 2003

17. Domain Name System (DNS)
• BIND 9.8.x
• Forward zone
– AAAA record for MX & selected Server
• Reverse zone
– PTR record for 2403:8000::/32 delegated from
APNIC

18. Web Server
• Apache Web Server 2.2.x
– Serve IPv4 and IPv6 at the same time
• IPv6 PHP script to detect v6 client
• Website IPv6 ITB
– http://www.itb.ac.id
– http://ipv6.itb.ac.id
– Masih banyak lagi

19. ITB Official Website

20. Mail Exchange (MX) Server
• Postfix 2.10
• mx.itb.ac.id
• http://www.postfix.org/IPV6_README.html

23. Web Cache Proxy Server
• Squid 2.7 (IPv4 only) and 3.1 (IPv6 support)
• Web Cache Parenting over IPv6
– to WIDE Project Japan
• Some IPv6 content observed
– Google IPv6
– Youtube IPv6
• Serving IPv6 client in ITB
• User Authentication with LDAP

24. Access.log Squid IPv6

26. Multicast Stream
• VLC
– IPv6 Unicast
– IPv6 Multicast
• Dokodemo SOI-ASIA
(http://dokodemo.soi.asia)

27. Dokodemo

29. IPv6 Day Activities
• Work together with SOI-ASIA
(http://ipv6day.soi.asia)
• IPv6-only video-on-demand streaming
– Adobe Flash Media Streaming Server on Linux
– 2 video of Indonesia cultural show
• IPv6-only website, embedding video
content
– http://ipv6day.itb.ac.id Apache on FreeBSD

30. Evaluating
IPv6 Server Load Balancer
• Provide IPv6 SLB for v6 client to v4 server
• IPv6 SLB that can translate:
– v6 client – v6 server
– v6 client – v6/v4 server
– v6 client – v4 server

31. Why IPv6 Load Balancer?
• To solve questions:
– Which is comes first, network or application?
– What is IPv6 killer apps?
• How it’s going to solve:
– IPv4 killer apps can directly migrated to IPv6
– No apps rewrite or migration
• At least in the theory
– Evaluation in the real world will tell you

32. Experience with IPv6 SLB (1)
• Basic services works just fine
• Translate IPv4 web server to IPv6 client
• Translate IPv4 cache server to IPv6 client
– real server(s) TCP4/8080 translated to virtual
IP on TCP6/8080
– virtual server client TCP6 server IPv6 client

33. Experience with IPv6 SLB (2)
• HTTP Layer 7 switching is mandatory
– or else cookie-based apps is not working
– Show stopper for webmaster to put webserver
behind SLB
• Managing SLB is quite hard for ordinary
network admin
– Lots of L7 feature to learn

34. Screenshot
Video
Stream
from
streaming
server
Simple
script to
identify
IPv6 client

35. Website Statistics (1)

36. Website Statistics (2)

37. User statistics
• Viewer observed from ITB campus
– Most of ITB campus network is IPv6 dual-stack
• Viewer also observed from Indonesia ISP
• Also observed from WIDE Project Japan
• No reverse address for IPv6
– It’s hard to see which ISP has IPv6 address
– Had to manually doing WHOIS on address

38. IPv6 tunnel broker for
Indonesia Universities
• Deployed on ITB router (Juniper SRX650)
– Ask INHERENT community to join
• Cleanup IPv6 prefix-list in TEIN3 ID-POP to
advertise new IPv6 prefix form
ITB/INHERENT

39. IPv6 BGP peering in ITB router
2001:470:17:72::1
6939
176059
23178
0
18
1w0d9h Establ
inet6.0: 5917/6690/6690/0
2001:7fa:f::1
7717
2907
2496
0
265
19:07:49 Establ
inet6.0: 118/123/123/0
2403:8000:10::2
18007
1
3
0
15950
1 Establ
inet6.0: 672/672/672/0
inet6.2: 79/79/79/0
2403:8000:666:dead::2
46047
2031
101953
0
149
16:53:49 Establ
inet6.0: 1/1/1/0
2403:8000:666:dead::6
55687
35
2699
0
123
20:57:09 Active
2403:8000:666:dead::a
45304
1
30432
0
9
9w4d1h Active
2403:8000:666:dead::18
46052
0
0
0
0
9w5d18h Idle
2403:8000:666:dead::22
55674
0
0
0
0
9w5d18h Active
• Red: IPv6 Tunnel BGP peering (AS6939 above is HE.net)
• Blue: IPv6 BGP peering to Indonesia OpenIXP
• Green: IPv6 BGP peering to INHERENT router to TEIN3
network

40. Statistics
• At least 5 tunnel registered, 3 of them observed
alive, only 1 currently active
• Unable to run IPv6 network monitoring, because
we haven’t setup the the infrastructure
• NetFlow v9 collector
• NFSen as NetFlow viewer

41. Hurricane Electric Tunnel
everywhere
• From simple show route protocol bgp, I see
most Indonesia ISP has HE.net tunnel
• AS6939 everywhere
– Makes BGP path adjustment difficult
• Path to AS6939 is preferred compared to TEIN3
• e.g., ITB needs to advertise /33 instead of /32 to TEIN3
– ITB has some IPv6 BGP peering
• Internet commercial IPv6 via HE.net

42. THE PRESENT

43. Status per 2013 (1)
• IPv6 stack is maturing:
– Router OS: Linux/BSD, Cisco, Juniper, Brocade,
HP, Huawei, Mikrotik, Force10, dsb
– Switch OS: Cisco Catalyst/Nexus, Juniper,
Brocade, HP, Huawei, Force10, dsb

44. Status per 2013 (2)
– Firewall: Cisco ASA, Juniper, Mikrotik, Palo Alto
– Load Balancer: F5 LTM, Brocade ADX, Apache
Traffic Server, Nginx, Varnish, Apache
mod_proxy module
– OS: Windows 7/8, Server 2008R2/2012, Mac
OS X, Linux/BSD
– Hypervisor: vSphere 5.x, RHEV, Hyper-V

45. Status per 2013 (3)
• OpenIXP provide IPv6 BGP
• Other ISP? Indosat? Telkom? Anyone?
• Temporary (permanently) solutions:
www.tunnelbroker.net
– bisa tunnel + peering BGP juga

46. However...
There are few things that gets in the way...

47. IPv6 without DNS =~ headache
• IPv6 address below is very hard to remember:
– 2403:8000:2e3b:6738:a573:c1bd:4b6c:31b7
•
•
•
•
Especially when you create IN PTR record
In order to use IPv6 network sniffer
In order to see access_log apache/squid
In order to see awstat/webalizer
• We should automate IN PTR creation in DNS

48. Happy Eyeball (1)
• Broken experience on IPv6 dual stack
means user won’t use IPv6
– https://ripe64.ripe.net/presentations/78-201204-16-ripe64.pdf dari Geoff Huston

49. Happy Eyeball (2)
• Need patch for all browser
• Most sysadmin choose to disable IPv6 for
end-user to mitigate complaints
• Or directry migrate to IPv6 only network
with NAT64/DNS64
– Small number of apps with literal IPv4
addressing won’t run

50. Slide happy eyeball

51. IPv6 Addressing scheme
• Or use existing IPv4 addressing scheme
– Easy to remember
• “Human-readable” IPv6 address:
– face:b00c (www.facebook.com)
– dead:beef
• IPAM is mandatory
– BlueCat Networks http://www.bluecatnetworks.com/ipam/
– GestioIP www.gestioip.net
– phpIPAM www.phpipam.net
• IPv6 Subnetting BCOP: http://www.ipbcop.org/ratifiedbcops/bcop-ipv6-subnetting/

52. Application guys don’t care
• They only care about their apps, without
knowing any networking property
– Managing responsive web, CSS and support
for IE6 is taking their time 
• Solution: IPv6 load balancer
– Dual stack SLB, IPv4-only web server
– Enable Layer 7 features, or else problems with
sticky apps
– Test your apps!

53. Security Issues
• Developing practices for IPv6
snort/IDS/IPS
• Port scanning is impossible
– You can’t run nmap -sP subnet/64
• Fragmentation attack
• RH0, source route
• Security compliance additional checklist

54. Bandwidth accounting
• How to inspect/police IPv6 bittorrent?
• Squid cache proxy
– Stable version don’t support IPv6 (2.7)
– IPv6 support in 3.2 is not as stable as 2.7
• Yes, you can put Squid behind IPv6 SLB
– But how about squid access log?
• This is problem in regular enterprise without
separate accounting/billing infra (telco)

55. User/client Provisioning
• DHCPv6 is not really like DHCPv4
• Two choices, which one to choose?
– IPv6 RA (ICMPv6) or DHCPv6?
• No DNS server record from IPv6 RA
– (you don't say?)
• Security issue in ICMPv6
– SEND = Secure ND

56. It feels like marathon
• Implementing IPv6 requires clear
milestone, resources and determination
• There are no deadline
• But sometimes you are out of resources 
– Our team members come and go
– Higher priority jobs gets in the way

57. THE FUTURE

58. What’s next for IPv6?
• Part of the ITB nextgen network blueprint
• IPv6 in hardware for all network devices
• Simpler transition mechanism
– NAT64/DNS64
– IPv6 SLB
• Simpler operation
– IPv6 full telemetry
– IPv6 address management

59. Roadmaps
Email
File
Sharing
Web
Hosting
HPC
Identity
Provider
Online
Learning
IS
OS/Hypervisor
Cloud
Orchestration
Platform as
a Service
Telepres
ence
Self service
Portal
Software as
a Service
BigData
Infrastructure
as a Service
Processor
Compute Cluster
IT
HPC
BigData
Cluster
Cluster
Cluster
Memory
Disk Storage
Storage Cluster
Live
Archive
Repo
Storage
Storage
Storage
I/O Inter
connect
Network Cluster
Core
Inter
Mgmt
Routing
connect

60. Network Blueprint

61. Networking for NGN Enterprise
• Basic IP routing
– IPv4/v6 unicast/multicast
– Policy-based routing/forwarding
• Advanced: MPLS on enterprise
– L3VPN, L2VPN, VPLS w/ TE/FRR
• Next generation network
– Ethernet fabric
– SDN: Software Defined Network
(programmable network)  OpenFlow

62. MPLS on Enterprise
• Enterprise ingin punya network yg flexible
seperti Telco
• Feature sets:
– L3VPN
– L2VPN
– VPLS
• High Availability
– MPLS TE
– FRR

63. MPLS Use Case for Campus
• L3VPN (IPv4 and IPv6, unicast & multicast)
– IP surveillance, RFID gate/reader, BMS
– Resell ISP bandwidth
• L2VPN
– Direct L2 connectivity from ISP
• VPLS
– Datacenter connectivity for cloud computing
– Single subnet wireless LAN deployment

64. IPv6 on all network devices (1)
• Router
– Unicast/multicast in Global Routing Table
– Unicast/multicast in VRF
• Firewall & NAT gateway
– IPv6 traffic inspection
– NAT64
• Server Load Balancer
– IPv6 SLB

65. IPv6 on all network devices (2)
• Network management infrastructure
– Devices telemetry: SNMP, Syslog
– Network telemetry: Netflow v9 / sFlow
– Authentication: RADIUS/Tacacs+
• Security management infrastructure
– Traffic inspection (IPS/IDS)
– Security Information & Event Management
(SIEM)

66. Simpler transition mechanism
• NAT64/DNS64 for IPv6-only network
– Good-enough IPv6-only experience
• IPv6 SLB for IPv4-only server
– Providing IPv6 content in an instant
• In the end, dual stack is not for everybody
– Only in network infrastructure
– Not good for endpoint

67. Simpler Operation
• IPAM (IP Address Management) is
mandatory
• In the future, tracking network resources
to IP address will not scale
– Track by User ID
– Track by application
– Track by content

68. Glimpse to the future:
SDN AND OPENFLOW

69. Software Defined Networking
(SDN)
In the SDN architecture, the control and data planes are
decoupled, network intelligence and state are logically
centralized, and the underlying network infrastructure is
abstracted from the applications.
Open Networking Foundation white paper
• OpenFlow is one of the SDN tool
– It’s the most popular ones

70. OpenFlow (1)
• Traditionally, control plane & forwarding
plane is integrated in same system
– Control plane: management, routing protocol
(OSPF, BGP) -> RIB, routing table
– Forwarding plane: packet forwarding -> FIB,
forwarding table
• SDN will decouple control plane function
to single controller

71. OpenFlow (2)
• Controller wil centrally manage routing for
the network
• Forwarding plane will forward the packet
based on decision from controller
– Forward, drop, send to controller, etc.
• Beberapa router menawarkan fitur
OpenFlow Hybrid Port
– One port/VLAN can simultaneously managed by
OpenFlow or by traditional routing protocol

72. Control/Data Plane Separation
•Control / Management plane in a dedicated controller
•Networking devices perform forwarding and maintenance functions
•IP / SSL connectivity between controller and OpenFlow switch
•OpenFlow = Forwarding table (TCAM) download protocol

73. Controller & Agents

74. Protocol Details

75. What’s so exciting about SDN?
• Sysadmin can centrally managed the
network without configuring each devices
• Sysadmin can program the network via
manual decision or automated, e.g. cloud
computing: OpenStack, VMware
• Flexibility above the traditional solution
• At least that’s the promise

76. Early SDN/OpenFlow Use Cases
• “Policy-based routing” or “packet filter”
• Replace traditional Layer 2 MAC learning
and propagation mechanisms
• Source:
– http://blog.ioshints.info/2011/11/openflowenterprise-use-cases.html
– http://datacenteroverlords.com/2011/11/07/openflow
-overlords/

77. And the challenges are...
• Building the network from scratch
– Event-driven network programming
– Fluency with TCP/IP layer
– Start learning now
• Things can fail massively

78. Troubleshooting gets complex
•
•
•
•
•
IGP/EGP routing -> RIB table
MPLS -> MPLS label table, VPN table
Also troubleshooting L2 is hard (VPLS, QinQ)
And there’s another one: SDN controller
You need to wrap around your head to
manage all of these abstraction

79. When should we adopt SDN?
• Start small, build virtual SDN labs
– OpenFlow controller
– Open vSwitch
• Evaluate SDN offering from vendors
• Collect SDN practices

80. CONCLUSION

81. Learned Lessons
• Put IPv6 as a requirement for next
generation network RFP
• Continuous milestone is essential to keep
IPv6 development under track
• Experience IPv6 operation early to
recognize pitfall and find solution

82. Reference
• Analysing Dual Stack Behaviour and IPv6 Quality – Geoff Huston &
George Michaelson - https://ripe64.ripe.net/presentations/78-201204-16-ripe64.pdf
• IPv6 Security – Scott Hogg & Eric Vyncke, Cisco Press http://www.amazon.com/IPv6-Security-Scott-Hogg/dp/1587055945
• NAT64 and DNS64 in 30 minutes – Ivan Pepelnjak ipSpace
http://blog.ioshints.info/2010/05/nat64-and-dns64-in-30minutes.html
• IPv6 Address Management – 6Help Australia
http://ipv6now.com.au/addresses.php
• OpenFlow and SDN: hype, useful tools or panacea? – Ivan Pepelnjak
- https://ripe65.ripe.net/presentations/19OpenFlow_and_SDN_(RIPE).pdf

83. Thanks!
It’s time to QA!