1. http://www.tech-invite.com
IPSec Guide
Architecture & Traffic Processing
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
V1.0 – March 2, 2005
This document presents the document roadmap for IPSec, as well as a host-to-
host architectural model, followed by a sequence of slides illustrating IPSec traffic
processing related to this model.
8 pages

2. IPSec Document Roadmap
RFC 2401
Security
RFC 2406 Architecture for the RFC 2402 RFC 2411
Internet Protocol
IP Encapsulating IP Authentication IP Security
Security Payload Stephen Kent Header Document
(ESP) Randall Atkinson (AH) Roadmap
Nov 1998
Stephen Kent Stephen Kent Rodney Thayer,
Randall Atkinson Randall Atkinson et. al.
Nov 1998 Nov 1998 Nov 1998
Uses Uses Uses
Dictate some of
Encryption Algorithms Authentication Algorithms
the values
RFC 2451 RFC 2410 RFC 2405 RFC 2403 RFC 2404 RFC 2104
ESP CBC-Mode NULL Encryption ESP DES-CBC Use of HMAC- Use of HMAC- HMAC: Keyed-
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
Cipher Algorithms Algorithm and its Cipher Algorithm RFC 2407 MD5-96 within SHA-1-96 within Hashing for
Use with IPsec with Explicit IV ESP and AH ESP and AH Message
Internet IP Security Authentication
Roy Pereira Rob Glenn Cheryl Madson Domain of Cheryl Madson Cheryl Madson
Rob Adams Stephen Kent N. Doraswamy Interpretation Rob Glenn Rob Glenn Krawczyk, et. al.
Nov 1998 Nov 1998 Nov 1998 (DOI) for ISAKMP Nov 1998 Nov 1998 Feb 1997
Derrell Piper
Nov 1998
Supplements IKE/ISAKMP with respect to Phase 2
RFC 2412
OAKLEY Key
Determination
RFC 2408 RFC 2409 Protocol SKEME
Internet Security Internet Key A versatile Secure
Association and Provides a framework Exchange Hilarie K. Orman Key Exchange
Key Management for authentication and (IKE) Uses parts of Nov 1998 Mechanism for
Protocol (ISAKMP) key exchange (but is not Internet
Dan Harkins dependant on)
Maughan, et. al. Dave Carrel these protocols Hugo Krawczyk
Nov 1998 Nov 1998 Nov 1995

3. IPSec Architecture – Host-to-Host Model
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
Domain-Wide
Policy Agent Policy Policy Agent
Manager
TCP/IP TCP/IP
Applications Applications
IP Main & Quick Main & Quick IP
Filters Modes Settings Modes Settings Filters
IKE ISAKMP SA IKE
UDP UDP
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
TCP / UDP TCP / UDP
SPD Security Policy Database
SPD SAD SAD Security Association Database SAD SPD
SA
AH AH
SA
IP IPSec IPSec IP
SA
ESP ESP
SA
IP@ a IP@ b
Network Network
Interface Interface

4. IPSec Traffic Processing – 1) Initialisation
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
1
Domain-Wide 1
Policy Agent Policy Policy Agent
3 2 Manager 2 3
TCP/IP TCP/IP
Applications Applications
IP Main & Quick Main & Quick IP
Filters Modes Settings Modes Settings Filters
1. Retrieve Policy Data from
a domain-wide manager
(as an alternative: from a
IKE local database)
IKE
2. Distribute security settings
UDP UDP
to IKE
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
3. Fill-in, directly or via the
IPSec Driver, the SPD
(Security Policy Database)
TCP / UDP with IP filters (ordered list TCP / UDP
of rules with selectors)
SPD SAD SAD SPD
AH AH
IP IPSec IPSec IP
ESP IP Connectivity ESP
between A and B
IP@ a IP@ b
is a prerequisite
Network Network
Interface Interface

5. IPSec Traffic Processing – 2) IKE Phase 1 Triggering
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
1. First outgoing quot;applicativequot; IP packet
2. The outgoing interface is IPSec-enabled and
therefore the packet is passed to the IPSec
driver
3. SPD Check returns quot;securedquot;
TCP/IP 4. Is there an appropriate active SA in SAD? - No TCP/IP
Applications 5. Request to IKE for creating the SA
Applications
6. IKE starts Phase 1 by sending an ISAKMP
message (quot;HDR, SAquot;) to Peer B
2. quot;IKEquot; IP packet passed to IPSec driver
3. SPD Check returns quot;permittedquot; (IKE traffic is
not to be secured via AH/ESP)
IKE 7. Packet returned unmodified by IPSec driver IKE
6 8. quot;IKEquot;Packet sent towards B 13
UDP 9. quot;IKEquot;Packet received by B UDP
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
10. The incoming interface is IPSec-enabled and
therefore the packet is passed to the IPSec
driver
TCP / UDP 11. SPD Check returns quot;permittedquot; (IKE traffic is TCP / UDP
not to be secured via AH/ESP)
SPD SAD 12. Packet returned by IPSec driver SAD SPD
13. IKE message received by IKE on side B
1 3 4 5 11
AH AH
2 12
IP IPSec IPSec IP
7 10
ESP ESP
IP@ a IP@ b
8 9
Network Network
Interface Interface

6. IPSec Traffic Processing – 3) IKE Phase 1 Completion
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
Note: the following exchanges are
detailed in another document
HDR, SA
Negotiation
TCP/IP Diffie-Hellman TCP/IP
Applications Exchange Applications
Authentication
IKE ISAKMP SA IKE
UDP UDP
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
TCP / UDP TCP / UDP
SPD SAD SAD SPD
AH AH
IP IPSec IPSec IP
ESP ESP
IP@ a IP@ b
Network Network
Interface Interface

7. IPSec Traffic Processing – 4) IKE Phase 2 & Secured Traffic Resumption
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
1. The Quick mode negotiation results in one 5. Packet is modified with ESP (depending on SA
outbound and one inbound SA, with SPI-a mode: transport / tunnel) and sent back to IP
and SPI-b respectively chosen by the 6. The secured packet is sent towards B
initiator and the responder 7. The secured packet is received on side B
2. The SAD is updated by IKE on each side 8. It is sent to the IPSec driver
TCP/IP 3. On the initiator side, IKE notifies the IPSec 9. The inbound SA is retrieved from the SPI value TCP/IP
Applications driver in answer to its previous request in the ESP header. Checkings are performed. Applications
4. Retrieval of SA parameters in the SAD is 10. The ESP header and trailer are removed and
resumed for the pending quot;applicationquot; the IP packet sent back to the IP module
packet 11. Payload is sent to upper layers
IKE ISAKMP SA IKE
1 1
UDP UDP
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
2 3 Note: the following exchanges are 2
detailed in another document
TCP / UDP 1 HDR*, HASH(1), TCP / UDP
SA, Ni...
SPD SAD HDR*, HASH(2), SAD SPD
SA, Nr...
4 HDR*, HASH(3) 9 11
AH AH
10
IP IPSec IPSec IP
5 10
5 SA 8
ESP ESP
SA
IP@ a IP@ b
6 1 1 7
Network Network
Interface Interface

8. IPSec Traffic Processing – 5) Secured (Outgoing) Traffic
IPSec Peer A (Initiator role) IPSec Peer B (Responder role)
TCP/IP TCP/IP
Applications Applications
IKE ISAKMP SA IKE
UDP UDP
#500 #500
Copyright © 2005-2007 Tech-invite.com Joël Repiquet. All Rights Reserved.
TCP / UDP TCP / UDP
SPD SAD SAD SPD
1 3 4 10 13
AH AH
2 12
IP IPSec IPSec IP
5 11
6 SA 9
ESP ESP
SA
IP@ a IP@ b
7 8
Network Network
Interface Interface