Internet Protocol Security (IPsec) protocol notes¶
1. Overview of IPsec¶
IPsec (Internet Protocol Security) is a suite of protocols that provides security services at the IP layer. It enables:
Confidentiality: Encryption of packet contents
Integrity: Verification that data hasn’t been modified in transit
Authentication: Confirmation of the identity of communicating peers
Anti-replay protection: Prevention of packet interception and retransmission
IPsec operates in two main modes:
Transport Mode: Protects upper-layer protocols (end-to-end)
Tunnel Mode: Protects entire IP packets (gateway-to-gateway)
2. Core IPsec protocols¶
Protocol |
Function |
Key Features |
|---|---|---|
AH (Authentication Header) |
Provides integrity and authentication |
Protocol 51, no encryption |
ESP (Encapsulating Security Payload) |
Provides encryption, integrity, and authentication |
Protocol 50, most commonly used |
IKE (Internet Key Exchange) |
Negotiates and manages security associations |
Uses UDP port 500 (IKEv1/v2) |
3. IPsec security associations (SAs)¶
Unidirectional logical connections between two peers
Defined by three parameters:
Security Parameter Index (SPI)
IP Destination Address
Security Protocol (AH or ESP)
Stored in Security Association Database (SAD)
Policy defined in Security Policy Database (SPD)
4. IKE phases¶
IKEv1:
Phase 1: Establishes secure channel (ISAKMP SA)
Main Mode (6 messages) or Aggressive Mode (3 messages)
Phase 2: Establishes IPsec SAs for data protection (Quick Mode)
IKEv2:
More efficient with fewer exchanges
Integrated NAT traversal
Built-in keepalives (dead peer detection)
5. Encryption and authentication algorithms¶
Algorithm Type |
Common Options |
Recommendations |
|---|---|---|
Encryption |
AES-GCM, AES-CBC, 3DES |
Prefer AES-GCM (128+ bits) |
Integrity |
SHA-256, SHA-384, SHA-1 |
Prefer SHA-256 or higher |
Diffie-Hellman Groups |
14 (2048-bit), 19 (ECP-256), 20 (ECP-384) |
Prefer ECP groups for efficiency |
PFS (Perfect Forward Secrecy) |
Enabled with DH key exchange |
Recommended for all deployments |
6. IPsec implementation models¶
Host-to-Host: Protection between two individual devices
Gateway-to-Gateway: Site-to-site VPN between networks
Host-to-Gateway: Remote access VPN (road warrior)
7. NAT traversal (NAT-T)¶
Allows IPsec to work through NAT devices
Uses UDP port 4500 for encapsulated ESP traffic
Automatically detected during IKE negotiation
8. Common IPsec issues & troubleshooting¶
Issue |
Symptoms |
Solutions |
|---|---|---|
Phase 1 failures |
No ISAKMP SA established |
Check pre-shared keys, authentication method |
Phase 2 failures |
ISAKMP SA ok, but no IPsec SA |
Check proxy IDs, encryption proposals |
NAT-T issues |
Connectivity behind NAT |
Ensure NAT-T enabled, UDP 4500 open |
MTU/fragmentation |
Performance issues, packet loss |
Adjust MTU, enable MSS clamping |
Routing issues |
Tunnels up but no traffic |
Check routing tables, policy routing |
9. Security considerations¶
Strong Pre-Shared Keys: Use long, complex keys or prefer certificate authentication
Algorithm Selection: Avoid weak algorithms (DES, MD5, SHA-1 when possible)
PFS Implementation: Always enable Perfect Forward Secrecy
Keepalives: Implement dead peer detection for connection stability
Logging/Monitoring: Monitor IPsec tunnels for failures and attacks
10. Quick configuration reference¶
Strong IKEv2 Proposal:
encryption: aes-256-gcm
integrity: n/a (GCM includes integrity)
dh-group: 20
prf: sha384
Strong IKEv1 Proposal:
encryption: aes-256-cbc
integrity: sha256
dh-group: 14
lifetime: 28800 seconds
ESP Proposal:
encryption: aes-256-gcm
lifetime: 3600 seconds
pfs: group20
11. IPsec vs SSL/TLS VPNs¶
Characteristic |
IPsec |
SSL/TLS |
|---|---|---|
Layer of Operation |
Network Layer (3) |
Application Layer (7) |
Client Requirements |
Often requires dedicated client |
Typically uses web browser |
Granularity |
Protects all IP traffic |
Protects specific applications |
NAT Traversal |
Requires NAT-T (UDP 4500) |
Works easily through NAT |
Implementation |
More complex to configure |
Generally easier to deploy |
Use this reference for designing, implementing, and troubleshooting IPsec VPNs.
Counter moves¶
Internet Protocol Security (IPsec) protocol notes is the variant in play. Strong IKE configuration and pruning weak proposals are the fix. Seen from the other side, this sits in the blue notes on traffic patterns as evidence.