IPsec is a set of protocols that encrypts and signs network traffic at the IP level, so every app gets protection without knowing it’s there.
Its main job today is the site-to-site VPN: linking two offices, or an office and a cloud network, over the public internet. It also powers the IKEv2 VPN option built into most phones and laptops.
We built an IPsec tunnel on our test machine to see what it hides. One setting left a payroll request readable to anyone watching.
What is IPsec?
IPsec, short for Internet Protocol Security, is an open standard from the IETF, the group that writes the internet’s rulebooks. It isn’t one protocol. It’s a toolkit of three parts that work together:
- IKE (Internet Key Exchange) lets two devices prove who they are and agree on fresh encryption keys. Version 2, IKEv2, is the one in use today.
- ESP (Encapsulating Security Payload) encrypts each packet and adds a check that catches any change in transit. This is what almost every IPsec VPN runs.
- AH (Authentication Header) signs packets so tampering shows, but encrypts nothing. You’ll rarely see it outside exams and old setups.
Because IPsec works at the network layer, it protects whole packets rather than one app’s connection. A browser, a database client and a printer driver all go through it the same way. That’s the big difference from HTTPS, which each app has to set up for itself.
The agreement between two devices is called a security association, or SA. It holds the keys and algorithms for one direction of traffic, so a working tunnel always has at least two. Each SA has an ID number, the SPI, stamped on every packet so the receiver knows which key to use.
How IPsec works, step by step
Say a branch office in Pune needs to reach the accounting server at head office in Mumbai. Each office has an IPsec gateway, usually its router or firewall. Here’s what happens when a laptop in Pune sends the first packet:
From first packet to protected traffic
What two IPsec gateways do before and during a connection
-
Step 1 of 6
A packet matches a policy
The Pune gateway checks its rules and sees the packet is headed for Mumbai’s private network. That traffic must be protected, so it holds the packet and starts IKE if no tunnel exists yet.
-
Step 2 of 6
The gateways agree on encryption
In the first IKEv2 exchange, the two sides pick algorithms, such as AES for encryption, and run a Diffie-Hellman key exchange. That gives both a shared secret without ever sending it across the internet.
-
Step 3 of 6
They prove who they are
Next, each side authenticates with a pre-shared key or a digital certificate. A gateway that can’t prove its identity gets no tunnel. IKEv2 does all of this in four messages.
-
Step 4 of 6
Child SAs are created
The gateways set up the SAs that carry real traffic, one per direction, each with its own SPI and keys.
-
Step 5 of 6
ESP wraps every packet
From now on, each packet bound for Mumbai is encrypted, given an integrity check and sent inside a new packet addressed gateway to gateway. Mumbai checks it, decrypts it and passes the original on.
-
Step 6 of 6
Keys are replaced on a timer
IKE swaps in fresh keys regularly, often every hour or so, so a key stolen later can’t unlock old traffic.
Our VPN tunnel explainer covers the wrapping idea in general terms. IPsec is one specific, standardised way of doing it.
What we saw when we built an IPsec tunnel
On 12 November we built a small test network on our Linux test machine. It had a laptop on a private office network, two gateways, a router playing the internet, and a private web server at the second site. The server replied with the address it saw.
First, with no IPsec, the laptop couldn’t reach the server at all. Private 10.x addresses don’t cross the internet, which is the problem site-to-site VPNs exist to solve.
Then we set up an ESP tunnel between the gateways with AES-GCM encryption. While the laptop requested a page with a fake session token in it, we captured the internet link:
server replied: seen-from 10.1.0.10 (the laptop's private address)
192.0.2.1 > 198.51.100.1: ESP(spi=0x00001001,seq=0x1), length 96
...
ESP packets captured: 12
plaintext 'session' matches: 0
inner 10.x addresses visible: 0The server saw the laptop’s real private address, so the two sites behaved like one network. The internet in between saw only the two gateways’ public addresses and numbered ESP packets. It couldn’t tell which laptop or which server was talking, let alone what they said.
The same test with AH
Next we switched the tunnel to AH, which signs packets without encrypting them. The capture told a very different story:
192.0.2.1 > 198.51.100.1: AH(spi=0x00001001,seq=0x3,...): IP 10.1.0.10 > 10.2.0.80.80:
HTTP: GET /payroll?session=abc123 HTTP/1.1
AH packets captured: 12
plaintext 'session' matches: 2
inner 10.x addresses visible: 12Anyone on the path could read the full request, token included. AH did its own job, though. When we gave the second gateway the wrong key, every packet was rejected and the request failed, with the kernel logging 3 failed integrity checks.
Fair test note: we set the keys by hand with the Linux kernel’s built-in IPsec support. So this lab skips the IKE negotiation a real gateway or VPN app would run. It shows what ESP and AH do to packets on the wire, not how fast a commercial IPsec VPN is.
One more number from the lab: a 1,000-byte packet left the tunnel as a 1,056-byte ESP packet. Those 56 bytes are the new IP header, the ESP header, and the integrity check. On a 1,500-byte link that overhead is why IPsec setups often lower the maximum packet size to avoid fragmented packets.
Tunnel mode vs transport mode
IPsec can wrap traffic two ways, and the choice decides what an observer can still see.
Tunnel mode encrypts the entire original packet, headers and all, and puts it inside a new packet between the two gateways. That’s what we ran, and it’s why the capture showed no private addresses. Site-to-site and remote-access VPNs use tunnel mode.
Transport mode keeps the original IP header and encrypts only the data behind it. It’s lighter, but anyone watching sees exactly which two machines are talking. You’ll find it protecting traffic between two servers that already have direct routes, or carrying an L2TP tunnel in the older L2TP/IPsec setup.
ESP vs AH: which one actually protects you?
For privacy, only ESP. AH has a second problem too: it signs the outer IP header. Home routers and mobile networks rewrite that header all the time, which breaks the signature. ESP leaves the outer header out of its check and works fine behind those routers.
| ESP | AH | |
|---|---|---|
| Encrypts the data | Yes | No |
| Detects tampering | Yes | Yes |
| Hides inner addresses (tunnel mode) | Yes | No |
| Works through address-translating routers | Yes, over UDP 4500 | No |
| IP protocol number | 50 | 51 |
| Seen in real VPNs | Almost always | Rarely |
Ports and protocols IPsec uses
If an IPsec VPN won’t connect, a firewall blocking one of these is the usual reason:
- UDP 500 for the IKE negotiation.
- UDP 4500 when either side sits behind address translation. IKE moves here and ESP packets ride inside UDP, a trick called NAT traversal.
- IP protocol 50 (ESP) for the encrypted traffic when there’s no address translation. It isn’t TCP or UDP, so it has no port number.
Hotel and guest Wi-Fi often allows only web traffic. There IPsec struggles, while a VPN running over TCP 443 gets through. Older routers needed a setting for this, explained in our VPN passthrough guide.
Wondering whether you need a VPN or a proxy at all? Our proxy vs VPN guide explains the difference.
IPsec vs WireGuard vs OpenVPN
IPsec isn’t the only way to build a VPN tunnel. Here’s how it compares with the two other protocols you’ll meet in VPN settings:
| IPsec (IKEv2) | WireGuard | OpenVPN | |
|---|---|---|---|
| Built into Windows, macOS, iOS, Android | Yes | No, needs an app | No, needs an app |
| Supported by business routers and firewalls | Nearly all | Some | Some |
| Setup effort | High, many options | Low | Medium |
| Gets through web-only firewalls | Poorly | Poorly (UDP only) | Yes, over TCP 443 |
| Handles switching Wi-Fi to mobile data | Yes (MOBIKE) | Yes | Reconnects |
For linking offices or cloud networks, IPsec is the default because every firewall vendor and cloud provider speaks it. For a personal VPN on a phone, WireGuard is usually faster and simpler. Our WireGuard vs OpenVPN comparison has speed numbers from our own tests.
Where you’ll actually run into IPsec
- Office-to-office and office-to-cloud links. A company with three branches and a cloud network typically runs an IPsec tunnel from each branch firewall. Bigger setups end all those tunnels on one box, a VPN concentrator.
- Work VPNs on laptops. Many corporate remote-access VPNs use IKEv2, sometimes through the operating system’s built-in client with no extra app. Our list of VPNs for remote work covers business options.
- Consumer VPN apps. Big providers such as NordVPN and Surfshark still offer IKEv2 as a protocol choice, handy on phones that hop between networks.
- L2TP/IPsec. An older combination that’s still in some router menus. If you’re given a choice, IKEv2 does the same job with less overhead.
Is IPsec secure?
Yes, when it’s set up with current settings. IPsec with IKEv2, AES and certificate or strong-key authentication protects banks, governments and cloud providers every day.
The weak spots come from configuration, not the design:
- Short pre-shared keys. A guessable shared password is the most common hole. Use a long random key or, better, certificates.
- Old algorithms left switched on. 3DES, SHA-1 and small Diffie-Hellman groups linger in older configs for compatibility. Turn them off.
- IKEv1 aggressive mode. This legacy mode exposes material an attacker can use to guess a pre-shared key offline. Use IKEv2.
- Too many options. IPsec has dozens of settings, and both sides must match. Mismatches cause most failed tunnels, and rushed fixes cause the insecure ones.
Like any VPN, IPsec also stops protecting you the moment the tunnel drops, unless your client blocks traffic until it’s back. That’s what a VPN kill switch does.
Frequently asked questions
Is IPsec a VPN?
Not by itself. IPsec is a set of security protocols, and a VPN is what you build with them. “An IPsec VPN” means a VPN that uses IPsec in tunnel mode to encrypt traffic between two gateways, or between a device and a gateway.
What layer of the OSI model does IPsec work at?
Layer 3, the network layer, the same layer as IP itself. That’s why it protects every app at once. TLS and SSL work higher up, at the session or transport layer, so each app has to use them on its own connections.
Is IKEv2 the same as IPsec?
Close, but not quite. IKEv2 is the part of IPsec that negotiates keys and authenticates both sides. The traffic itself is then protected by ESP. When a VPN app lists “IKEv2” or “IKEv2/IPsec” as a protocol, both are running together.
Does IPsec use TCP or UDP?
Neither for the data, normally. IKE runs over UDP 500, and ESP is its own IP protocol, number 50. When there’s address translation in the path, both move inside UDP 4500. Some vendors can also wrap IPsec in TCP to pass strict firewalls.
Is IPsec faster than WireGuard?
Usually not on typical hardware, though the gap depends on the device. Many business firewalls and some CPUs have hardware that speeds up IPsec encryption, which can close or reverse the gap there. On a phone or laptop, WireGuard is generally quicker to connect and lighter.
Can IPsec be blocked?
Yes, fairly easily. Its UDP ports 500 and 4500 and its ESP packets are easy to spot, so a firewall that allows only web traffic stops it. VPNs that run over TCP 443, such as OpenVPN in TCP mode, get through those networks more often.
Do I need IPsec for home use?
Probably not directly. If you use a consumer VPN app, pick whichever protocol is fastest for you, usually WireGuard. IPsec matters at home mainly if your employer’s VPN uses it, or if you want to link your home network to a cloud server.
The bottom line
So, what is IPsec? It’s the standard way to encrypt traffic between whole networks, so separate sites can act like one private network over the internet.
Linking sites or setting up a firewall? IPsec with IKEv2 and ESP in tunnel mode is the default. Use certificates or long keys, and turn off old algorithms. If you just want a private connection on your phone, WireGuard is usually the simpler pick.
