Exim Specification - 49. Message processing

Go to the first, previous, next, last section, table of contents.

# 49. Message processing

Exim performs various transformations on the sender and recipient addresses of all messages that it handles, and also on the messages' header lines. Some of these are optional and configurable, while others always take place. All of this processing, except rewriting as a result of routing, happens when a message is received, before it is first written to the spool.

RFC 822 makes provision for headers starting with the string ``Resent-'`. It states that in general, the ``Resent-'` fields should be treated as containing a set of information that is independent of the set of original fields, and that information for one set should not automatically be taken from the other. If Exim finds any ``Resent-'` headers in the message, it applies the header transformations described below only to the ``Resent-'` header set, leaving the unqualified set alone.

## 49.1 Unqualified addresses

By default, Exim expects every address it receives from an external host to be fully qualified. Unqualified addresses cause negative responses to SMTP commands. However, because SMTP is used as a means of transporting messages from MUAs running on personal workstations, there is sometimes a requirement to accept unqualified addresses from specific hosts or IP networks.

Exim has two options that separately control which hosts may send unqualified sender or receiver addresses in SMTP commands, namely sender_unqualified_hosts and receiver_unqualified_hosts. In both cases, if an unqualified address is accepted, it is qualified by adding the value of qualify_domain or qualify_recipient, as appropriate.

Unqualified addresses are accepted only from local sources or from hosts that match one of the receiver_unqualified or sender_unqualified options, as appropriate.

## 49.2 The UUCP From line

Messages that have come from UUCP (and some other applications) often begin with a line containing the envelope sender and a timestamp, following the word `From'. Examples of two common formats are:

    From a.oakley@berlin.mus Fri Jan  5 12:35 GMT 1996
    From f.butler@berlin.mus Fri, 7 Jan 97 14:00:00 GMT

This line precedes the RFC 822 header lines. For compatibility with Sendmail, Exim recognizes such lines at the start of messages that are submitted to it via the command line (that is, on the standard input). It does not recognize such lines in incoming SMTP messages, unless the sending host matches ignore_fromline_hosts or the -bs option was used for a local message and ignore_fromline_local is set. The recognition is controlled by a regular expression that is defined by the uucp_from_pattern option, whose default value matches the two common cases shown above and puts the address that follows `From' into $1.

When the caller of Exim for a non-SMTP message is a trusted user, the message's sender address is constructed by expanding the contents of uucp_sender_address, whose default value is `$1'. This is then parsed as an RFC 822 address. If there is no domain, the local part is qualified with qualify_domain unless it is the empty string. However, if the command line -f option is used, it overrides the `From' line.

If the caller of Exim is not trusted, the `From' line is recognized, but the sender address is not changed. This is also the case for incoming SMTP messages that are permitted to contain `From' lines.

Only one `From' line is recognized. If there is more than one, the second is treated as a data line that starts the body of the message, as it is not valid as a header line. This also happens if a `From' line is present in an incoming SMTP message from a source that is not permitted to send them.

## 49.3 The Bcc header

If Exim is called with the -t option, to take recipient addresses from a message's headers, it removes any Bcc: header that may exist (after extracting its addresses), unless the message has no To: or Cc: header, in which case a Bcc: header with no addresses is left in the message, in accordance with RFC 822. If -t is not present on the command line, any existing Bcc: header is not removed.

If Exim is called to receive a message with the recipient addresses given on the command line, and there is no Bcc:, To:, or Cc: header in the message, it normally adds a To: header, listing the recipients. Some mailing list software is known to submit messages in this way, and in this case the creation of a To: header is not what is wanted. If the always_bcc option is set, Exim adds an empty Bcc: header instead in this circumstance.

## 49.4 The Date header

If a message has no Date: header, Exim adds one, giving the current date and time.

## 49.5 The Delivery-date header

Delivery-date: headers are not part of the standard RFC 822 header set. Exim can be configured to add them to the final delivery of messages. (See the generic delivery_date_add transport option.) They should not be present in messages in transit. If the delivery_date_remove configuration option is set (the default), Exim removes Delivery-date: headers from incoming messages.

## 49.6 The Envelope-to header

Envelope-to: headers are not part of the standard RFC 822 header set. Exim can be configured to add them to the final delivery of messages. (See the generic envelope_to_add transport option.) They should not be present in messages in transit. If the envelope_to_remove configuration option is set (the default), Exim removes Envelope-to: headers from incoming messages.

## 49.7 The From header

If an incoming message does not contain a From: header, Exim adds one containing the sender's address. This is obtained from the message's envelope in the case of remote messages; for locally-generated messages the calling user's login name and full name are used to construct an address, as described in section 49.14. They are obtained from the password file entry by calling getpwuid() (but see the unknown_login configuration option). The address is qualified with qualify_domain.

For compatibility with Sendmail, if an incoming, non-SMTP message has a From: header containing just the unqualified login name of the calling user, this is replaced by an address containing the user's login name and full name as described in section 49.14.

## 49.8 The Message-id header

If an incoming message does not contain a Message-id: header, Exim constructs one and adds it to the message. The id is constructed from Exim's internal message id, preceded by the letter E to ensure it starts with a letter, and followed by @ and the primary host name. Additional information can be included in this header by setting the message_id_header_text option.

## 49.9 The Received header

A Received: header is added at the start of every message. The contents of this header are defined by the received_header_text configuration option, and Exim automatically adds a semicolon and a timestamp to the configured string.

## 49.10 The Return-path header

Return-path: headers are defined as something the MTA may insert when it does the final delivery of messages. (See the generic return_path_add transport option.) Therefore, they should not be present in messages in transit. If the return_path_remove configuration option is set (the default), Exim removes Return-path: headers from incoming messages.

## 49.11 The Sender header

For locally-originated messages, unless originated by a trusted user, any existing Sender: header is removed. For non-trusted callers, unless local_from_check is set false, a check is made to see if the address given in the From: header is the correct (local) sender of the message (prefixes and suffixes for the local part can be permitted via local_from_prefix and local_from_suffix). If not, a Sender: header giving the true sender address is added to the message. No processing of the Sender: header is done for messages originating externally.

## 49.12 The To header

If a message has no To:, Cc:, or Bcc: header, Exim adds an empty Bcc: header, in accordance with RFC 822, except when the message is being received locally with the recipients supplied on the command line. In this case, a To: header listing the recipients is normally added. Some mailing list software is known to submit messages in this way, and in this case the creation of a To: header is not what is wanted. If the always_bcc option is set, Exim adds an empty Bcc: header instead in this circumstance. An Apparently-to: header is never added.

## 49.13 Adding and removing headers

The addition and removal of headers can be specified on any of the drivers, and also in system filter files. Changes specified in the system filter affect all deliveries of a message.

Header changes specified on a director or router affect all addresses handled by that driver, and also any new addresses it generates. If an address passes through several directors and/or routers, the changes are cumulative. When a message is processed by a transport, the message's original set of headers is output, except for those named in any headers_remove options that the address has encountered as it was processed, and any in the transport's own headers_remove option. Then any new headers from any headers_add options are output.

## 49.14 Constructed addresses

When Exim constructs a sender address for a locally-generated message, it uses the form

    <user name> <<login>@<qualify_domain>>

For example:

    Zaphod Beeblebrox <zaphod@end.univ>

The user name is obtained from the -F command line option if set, or otherwise by looking up the calling user by getpwuid() and extracting the `gecos' field from the password entry. If the `gecos' field contains an ampersand character, this is replaced by the login name with the first letter upper-cased, as is conventional in a number of operating systems. See the gecos_name option for a way to tailor the handling of the `gecos' field. The unknown_username option can be used to specify user names in cases when there is no password file entry.

In all cases, the user name is made to conform to RFC 822 by quoting all or parts of it if necessary. In addition, if it contains any non-printing characters, it is encoded as described in RFC 2047, which defines a way of including non-ASCII characters in header lines. The setting of print_topbitchars controls whether characters with the top bit set (that is, with codes greater than 127) count as printing characters or not.

## 49.15 Case of local parts

RFC 822 states that the case of letters in the local parts of addresses cannot be assumed not to be significant. Exim preserves the case of local parts of remote addresses. However, when it is processing an address in one of its local domains, the case of letters in the local part is significant only when locally_caseless is unset. This option is set by default, and this causes Exim to lowercase local parts in local domains before processing them.

If you must have mixed-case user names in your password file, the best way to proceed, assuming you want case-independent handling of incoming email, is to unset locally_caseless and then set up an initial smartuser director to convert incoming local parts to the correct case by a file lookup such as

    new_address = ${lookup{${lc:$local_part}}cdb\
                  {/etc/usercased.cdb}{$value}fail}\
                  @$domain

## 49.16 Dots in local parts

RFC 822 forbids empty components in local parts. That is, an unquoted local part may not begin or end with a dot, nor have two consecutive dots in the middle. However, it seems that many MTAs do not enforce this, so Exim permits empty components for compatibility.

## 49.17 Rewriting addresses

Rewriting of sender and recipient addresses, and addresses in headers, can happen automatically, or as the result of configuration options, as described in chapter 34. The headers that may be affected by this are Bcc:, Cc:, From:, Reply-To:, Sender:, and To:.

Automatic rewriting includes qualification, as mentioned above. The other case in which it can happen is when an incomplete non-local domain is given. The routing process may cause this to be expanded into the full domain name. For example, a header such as

    To: hare@teaparty

might get rewritten as

    To: hare@teaparty.wonderland.fict.book

Rewriting as a result of routing is the one kind of message processing that does not happen at input time, as it cannot be done until the address has been routed.

Strictly, one should not do any deliveries of a message until all its addresses have been routed, in case any of the headers get changed as a result of routing. However, doing this in practice would hold up many deliveries for unreasonable amounts of time, just because one address could not immediately be routed. Exim therefore does not delay other deliveries when routing of one or more addresses is deferred.

Go to the first, previous, next, last section, table of contents.