The IPsec protocols

This section provides details of the IPsec protocols which FreeS/WAN implements

The basic idea of IPsec is to provide security functions, authentication and encryption, at the IP (Internet Protocol) level. This requires a higher-level protocol (IKE) to set things up for the IP-level services (ESP and AH).

Three protocols are used in an IPsec implementation:

ESP, Encapsulating Security Payload
Encrypts and/or authenticates data
AH, Authentication Header
Provides a packet authentication service
IKE, Internet Key Exchange
Negotiates connection parameters, including keys, for the other two

The term "IPsec" (also written as IPSEC) is slightly ambiguous. In some contexts, it includes all three of the above but in other contexts it refers only to AH and ESP.

Applying IPsec

Authentication and encryption functions for network data can, of course, be provided at other levels. Many security protocols work at levels above IP.

and so on. Other techniques work at levels below IP. For example, data on a communications circuit or an entire network can be encrypted by specialised hardware. This is common practice in high-security applications.

Advantages of IPsec

There are, however, advantages to doing it at the IP level instead of, or as well as, at other levels.

IPsec is the most general way to provide these services for the Internet.

IPsec, however, can protect any protocol running above IP and any medium which IP runs over. More to the point, it can protect a mixture of application protocols running over a complex combination of media. This is the normal situation for Internet communication; IPsec is the only general solution.

IPsec can also provide some security services "in the background", with no visible impact on users. To use PGP encryption and signatures on mail, for example, the user must at least:

These systems can be designed so that the burden on users is not onerous, but any system will place some requirements on users. No such system can hope to be secure if users are sloppy about meeting those requirements. The author has seen username and password stuck on terminals with post-it notes in an allegedly secure environment, for example.

Limitations of IPsec

IPsec is designed to secure IP links between machines. It does that well, but it is important to remember that there are many things it does not do. Some of the important limitations are:

IPsec cannot be secure if your system isn't
System security on IPsec gateway machines is an essential requirement if IPsec is to function as designed. No system can be trusted if the underlying machine has been subverted. See books on Unix security such as Garfinkel and Spafford or our web references for Linux security or more general computer security.

Of course, there is another side to this. IPsec can be a powerful tool for improving system and network security. For example, requiring packet authentication makes various spoofing attacks harder and IPsec tunnels can be extremely useful for secure remote administration of various things.

IPsec is not end-to-end
IPsec cannot provide the same end-to-end security as systems working at higher levels. IPsec encrypts an IP connection between two machines, which is quite a different thing than encrypting messages between users or between applications.

For example, if you need mail encrypted from the sender's desktop to the recipient's desktop and decryptable only by the recipient, use PGP or another such system. IPsec can encrypt any or all of the links involved -- between the two mail servers, or between either server and its clients. It could even be used to secure a direct IP link from the sender's desktop machine to the recipient's, cutting out any sort of network snoop. What it cannot ensure is end-to-end user-to-user security. If only IPsec is used to secure mail, then anyone with appropriate privileges on any machine where that mail is stored (at either end or on any store-and-forward servers in the path) can read it.

In another common setup, IPsec encrypts packets at a security gateway machine as they leave the sender's site and decrypts them on arrival at the gateway to the recipient's site. This does provide a useful security service -- only encrypted data is passed over the Internet -- but it does not even come close to providing an end-to-end service. In particular, anyone with appropriate privileges on either site's LAN can intercept the message in unencrypted form.

IPsec cannot do everything
IPsec also cannot provide all the functions of systems working at higher levels of the protocol stack. If you need a document electronically signed by a particular person, then you need his or her digital signature and a public key cryptosystem to verify it with.

Note, however, that IPsec authentication of the underlying communication can make various attacks on higher-level protocols more difficult. In particular, authentication prevents man-in-the-middle attacks.

IPsec authenticates machines, not users
IPsec uses strong authentication mechanisms to control which messages go to which machines, but it does not have the concept of user ID, which is vital to many other security mechansims and policies. This means some care must be taken in fitting the various security mechansims on a network together. For example, if you need to control which users access your database server, you need some non-IPsec mechansim for that. IPsec can control which machines connect to the server, and can ensure that data transfer to those machines is done securely, but that is all. Either the machines themselves must control user access or there must be some form of user authentication to the database, independent of IPsec.

IPsec does not stop denial of service attacks
Denial of service attacks aim at causing a system to crash, overload, or become confused so that legitimate users cannot get whatever services the system is supposed to provide. These are quite different from attacks in which the attacker seeks either to use the service himself or to subvert the service into delivering incorrect results.

IPsec shifts the ground for DoS attacks; the attacks possible against systems using IPsec are different than those that might be used against other systems. It does not, however, eliminate the possibility of such attacks.

IPsec does not stop traffic analysis
Traffic analysis is the attempt to derive intelligence from messages without regard for their contents. In the case of IPsec, it would mean analysis based on things visible in the unencrypted headers of encrypted packets -- source and destination gateway addresses, packet size, et cetera. Given the resources to acquire such data and some skill in analysing it (both of which any national intelligence agency should have), this can be a very powerful technique.

IPsec is not designed to defend against this. Partial defenses are certainly possible, and some are described below, but it is not clear that any complete defense can be provided.

IPsec is a general mechanism for securing IP

While IPsec does not provide all functions of a mail encryption package, it can encrypt your mail. In particular, it can ensure that all mail passing between a pair or a group of sites is encrypted. An attacker looking only at external traffic, without access to anything on or behind the IPsec gateway, cannot read your mail. He or she is stymied by IPsec just as he or she would be by PGP.

The advantage is that IPsec can provide the same protection for anything transmitted over IP. In a corporate network example, PGP lets the branch offices exchange secure mail with head office. SSL and SSH allow them to securely view web pages, connect as terminals to machines, and so on. IPsec can support all those applications, plus database queries, file sharing (NFS or Windows), other protocols encapsulated in IP (Netware, Appletalk, ...), phone-over-IP, video-over-IP, ... anything-over-IP. The only limitation is that IP Multicast is not yet supported, though there are Internet Draft documents for that.

IPsec creates secure tunnels through untrusted networks. Sites connected by these tunnels form VPNs, Virtual Private Networks.

IPsec gateways can be installed wherever they are required.

Which of these, or of the many other possible variants, to use is up to you. IPsec provides mechanisms; you provide the policy.

No end user action is required for IPsec security to be used; they don't even have to know about it. The site administrators, of course, do have to know about it and to put some effort into making it work. Poor administration can compromise IPsec as badly as the post-it notes mentioned above. It seems reasonable, though, for organisations to hope their system administrators are generally both more security-conscious than end users and more able to follow computer security procedures. If not, at least there are fewer of them to educate or replace.

IPsec can be, and often should be, used with along with security protocols at other levels. If two sites communicate with each other via the Internet, then IPsec is the obvious way to protect that communication. If two others have a direct link between them, either link encryption or IPsec would make sense. Choose one or use both. Whatever you use at and below the IP level, use other things as required above that level. Whatever you use above the IP level, consider what can be done with IPsec to make attacks on the higher levels harder. For example, man-in-the-middle attacks on various protocols become difficult if authentication at packet level is in use on the potential victims' communication channel.

Using authentication without encryption

Where appropriate, IPsec can provide authentication without encryption. One might do this, for example:

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

Encryption without authentication is dangerous

Originally, the IPsec encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPsec working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.
Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPsec specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

Multiple layers of IPsec processing are possible

The above describes combinations possible on a single IPsec connection. In a complex network you may have several layers of IPsec in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPsec tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPsec applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns about any of those, then he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPsec is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPsec and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

Using multiple encryption

This also applies to using multiple layers of encryption. If you have an IPsec tunnel between two branch offices, it c tunnehose protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

Encryption without authentication is dangerous

Originally, the IPsec encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPsec working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.
Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPsec specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

Multiple layers of IPsec processing are possible

The above describes combinations possible on a single IPsec connection. In a complex network you may have several layers of IPsec in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPsec tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPsec applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns about any of those, then he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPsec is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPsec and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

Using multiple encryption

This also applies to using multiple layers of encryption. If you have an IPsec tunnel between two branch offices, it c tunnehose protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

Encryption without authentication is dangerous

Originally, the IPsec encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPsec working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.
Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPsec specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

Multiple layers of IPsec processing are possible

The above describes combinations possible on a single IPsec connection. In a complex network you may have several layers of IPsec in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPsec tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPsec applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns about any of those, then he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPsec is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPsec and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

Using multiple encryption

This also applies to using multiple layers of encryption. If you have an IPsec tunnel between two branch offices, it c tunnehose protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

Encryption without authentication is dangerous

Originally, the IPsec encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPsec working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.
Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPsec specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

Multiple layers of IPsec processing are possible

The above describes combinations possible on a single IPsec connection. In a complex network you may have several layers of IPsec in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPsec tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPsec applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns about any of those, then he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPsec is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPsec and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

Using multiple encryption

This also applies to using multiple layers of encryption. If you have an IPsec tunnel between two branch offices, it c tunnehose protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further along.

Encryption without authentication is dangerous

Originally, the IPsec encryption protocol ESP didn't do integrity checking. It only did encryption. Steve Bellovin found many ways to attack ESP used without authentication. See his paper Problem areas for the IP Security Protocols. To make a secure connection, you had to add an AH Authentication Header as well as ESP. Rather than incur the overhead of several layers (and rather than provide an ESP layer that didn't actually protect the traffic), the IPsec working group built integrity and replay checking directly into ESP.

Today, typical usage is one of:

Other variants are allowed by the standard, but not much used:

ESP encryption without authentication
Bellovin has demonstrated fatal flaws in this. Do not use.
ESP encryption with AH authentication
This has higher overheads than using the authentication in ESP, and no obvious benefit in most cases. The exception might be a network where AH authentication was widely or universally used. If you're going to do AH to conform with network policy, why authenticate again in the ESP layer?
Authenticate twice, with AH and with ESP
Why? Of course, some folk consider "belt and suspenders" the sensible approach to security. If you're among them, you might use both protocols here. You might also use both to satisfy different parts of a security policy. For example, an organisation might require AH authentication everywhere but two users within the organisation might use ESP as well.
ESP authentication without encryption
The standard allows this, calling it "null encryption". FreeS/WAN does not support it. We recommend that you use AH instead if authentication is all you require. AH authenticates parts of the IP header, which ESP-null does not do.
Some of these variants cannot be used with FreeS/WAN because we do not support ESP-null and do not support automatic keying of AH-only connections.

There are fairly frequent suggestions that AH be dropped entirely from the IPsec specifications since ESP and null encryption can handle that situation. It is not clear whether this will occur. My guess is that it is unlikely.

Multiple layers of IPsec processing are possible

The above describes combinations possible on a single IPsec connection. In a complex network you may have several layers of IPsec in play, with any of the above combinations at each layer.

For example, a connection from a desktop machine to a database server might require AH authentication. Working with other host, network and database security measures, AH might be just the thing for access control. You might decide not to use ESP encryption on such packets, since it uses resources and might complicate network debugging. Within the site where the server is, then, only AH would be used on those packets.

Users at another office, however, might have their whole connection (AH headers and all) passing over an IPsec tunnel connecting their office to the one with the database server. Such a tunnel should use ESP encryption and authentication. You need authentication in this layer because without authentication the encryption is vulnerable and the gateway cannot verify the AH authentication. The AH is between client and database server; the gateways aren't party to it.

In this situation, some packets would get multiple layers of IPsec applied to them, AH on an end-to-end client-to-server basis and ESP from one office's security gateway to the other.

Resisting traffic analysis

Traffic analysis is the attempt to derive useful intelligence from encrypted traffic without breaking the encryption.

Is your CEO exchanging email with a venture capital firm? With bankruptcy trustees? With an executive recruiting agency? With the holder of some important patents? If an eavesdropper learns about any of those, then he has interesting intelligence on your company, whether or not he can read the messages themselves.

Except in the simplest cases, traffic analysis is hard to do well. It requires both considerable resources and considerable analytic skill. However, intelligence agencies of various nations have been doing it for centuries and many of them are likely quite good at it by now. Various commercial organisations, especially those working on "targeted marketing" may also be quite good at analysing certain types of traffic.

In general, defending against traffic analysis is also difficult. Inventing a really good defense could get you a PhD and some interesting job offers.

IPsec is not designed to stop traffic analysis and we know of no plausible method of extending it to do so. That said, there are ways to make traffic analysis harder. This section describes them.

Using "unnecessary" encryption

One might choose to use encryption even where it appears unnecessary in order to make analysis more difficult. Consider two offices which pass a small volume of business data between them using IPsec and also transfer large volumes of Usenet news. At first glance, it would seem silly to encrypt the newsfeed, except possibly for any newsgroups that are internal to the company. Why encrypt data that is all publicly available from many sites?

However, if we encrypt a lot of news and send it down the same connection as our business data, we make traffic analysis much harder. A snoop cannot now make inferences based on patterns in the volume, direction, sizes, sender, destination, or timing of our business messages. Those messages are hidden in a mass of news messages encapsulated in the same way.

If we're going to do this we need to ensure that keys change often enough to remain secure even with high volumes and with the adversary able to get plaintext of much of the data. We also need to look at other attacks this might open up. For example, can the adversary use a chosen plaintext attack, deliberately posting news articles which, when we receive and encrypt them, will help break our encryption? Or can he block our business data transmission by flooding us with silly news articles? Or ...

Also, note that this does not provide complete protection against traffic analysis. A clever adversary might still deduce useful intelligence from statistical analysis (perhaps comparing the input newsfeed to encrypted output, or comparing the streams we send to different branch offices), or by looking for small packets which might indicate establishment of TCP connections, or ...

As a general rule, though, one can improve resistance to traffic analysis by encrypting as much traffic as possible rather than only as much as seems necessary.

Using multiple encryption

This also applies to using multiple layers of encryption. If you have an IPsec tunnel between two branch offices, it c tunnehose protocols harder.

Authentication has lower overheads than encryption.

The protocols provide four ways to build such connections, using either an AH-only connection or ESP using null encryption, and in either manually or automatically keyed mode. FreeS/WAN supports only one of these, manually keyed AH-only connections, and we do not recommend using that. Our reasons are discussed under Resisting traffic analysis a few sections further al