dccm(8)               Distributed Checksum Clearinghouse               dccm(8)

NAME
     dccm -- Distributed Checksum Clearinghouse Milter Interface

SYNOPSIS
     dccm [-VdbxANQW] [-G on | off | noIP | IPmask/xx] [-h homedir]
          [-p protocol:filename | protocol:port@host] [-m map]
          [-w whiteclnt] [-U userdirs] [-a IGNORE | REJECT | DISCARD]
          [-t type,[log-thold,]rej-thold] [-g [not-]type] [-S header]
          [-l logdir] [-R rundir] [-r rejection-msg] [-j maxjobs]
          [-B dnsbl-option] [-L ltype,facility.level]

DESCRIPTION
     Dccm is a daemon built with the sendmail milter interface intended to
     connect sendmail to DCC servers.  When built with the milter filter
     machinery and configured to talk to dccm in the sendmail.cf file, send-
     mail passes all email to dccm which in turn reports related checksums to
     the nearest DCC server.  Dccm then adds an X-DCC SMTP header line to the
     message.  Sendmail is told to reject the message if it is unsolicited
     bulk mail.

     dccm sends reports of checksums related to mail received by DCC clients
     and queries about the total number of reports of particular checksums.  A
     DCC server receives no mail, address, headers, or other information, but
     only cryptographically secure checksums of such information.  A DCC
     server cannot determine the text or other information that corresponds to
     the checksums it receives.  Its only acts as a clearinghouse of counts
     for checksums computed by clients.  For complete privacy as far as the
     DCC is concerned, the checksums of purely internal mail or other mail
     that is known to not be unsolicited bulk can be listed in a whitelist to
     not be reported to the DCC server.

     Since the checksums of messages that are whitelisted locally by the -w
     whiteclnt file are not reported to the DCC server, dccm knows nothing
     about the total recipient counts for their checksums and so cannot add
     X-DCC header lines to such messages.  Sendmail does not tell dccm about
     messages that are not received by sendmail via SMTP, including messages
     submitted locally and received via UUCP, and so they also do not receive
     X-DCC header lines.

     The list of servers that dccm contacts is in a memory mapped file shared
     by local DCC clients.  The file is  maintained with cdcc(8).  Put parame-
     ters into the dcc_conf file and start the daemon with the start-dccm
     script.

     When sendmail is not used, then dccm is not useful.  dccproc(8) or
     dccifd(8) can often be used instead.

   OPTIONS
     The following options are available:

     -V   displays the version of the DCC Milter interface.

     -d   enables debugging output from the DCC client library.  Additional -d
          options increase the number of messages.  A single -d
           aborted SMTP transactions including those from some "dictionary
          attacks."

     -b   causes the daemon to not detach itself from the controlling tty and
          put itself into the background.

     -x   causes the daemon to try "extra hard" to contact a DCC server.
          Since it is usually more important to deliver mail than to report
          its checksums, dccm normally does not delay too long while trying to
          contact a DCC server.  It will not try again for several seconds
          after a failure.  With -x, unresponsive DCC servers cause mail to be
          temporarily rejected with 4.7.1 451 DCC failure

     -A   adds to existing X-DCC headers in the message instead of replacing
          existing headers of the brand of the current server.

     -N   neither adds, deletes, nor replaces existing X-DCC headers in the
          message.  Each message is logged, rejected, and otherwise handled
          the same.

     -Q   only queries the DCC server about the checksums of messages instead
          of reporting and querying.  This is useful when dccm is used to fil-
          ter mail that has already been reported to a DCC server by another
          DCC client.  This can also be useful when applying a private white
          or black list to mail that has already been reported to a DCC
          server.  No single mail message should be reported to a DCC server
          more than once per recipient, because each report will increase the
          apparent "bulkness" of the message.

     -G on | off | noIP | IPmask/xx
          controls greylisting.  At least one working greylist server must be
          listed in the map file in the DCC home directory.  If more than one
          is named, they must "flood" or change checksums and they must use
          the same -G parameters.  See dccd(8).  Usually all DCC client pro-
          cesses of dccm or dccifd should use the same -G parameters.

          IPmask/xx and noIP remove part or all of the IP address from the
          greylist triple.  The CIDR block size, xx, must be between 1 and
          128.  96 is added to block sizes smaller than 33 to make them appro-
          priate for the IPv6 addresses used by the DCC.  IPmask/96 differs
          from noIP because the former retains the IPv4 to IPv6 mapping pre-
          fix.

     -W   turns off DCC filtering by default to ease managing systems where
          only a minority of users want unsolicited bulk mail to be rejected
          or discarded.  This is equivalent to a option dcc-off line in the
          main -w whiteclnt file.  When DCC filtering is off, mail is handled
          as if -a IGNORE were in use.  The DCC server is queried and the
          X-DCC header is added but the message is delivered regardless of
          target counts and thresholds.

          DCC filtering is enabled for a mailbox when -W is not used and there
          is no option dcc-off line in the main or per-user whiteclnt file or
          there is a option dcc-on pine in the per-user whiteclnt file for the
          mailbox.  DCC filtering can also be enabled with an "OK2" entry for
          the fully qualified mailbox in the main or per-user whiteclnt file.

          Messages sent only to target addresses that are listed in the global
          or relevant per-user -w whiteclnt file with "OK" are not reported to
          the DCC server and so are not rejected or discarded and do not
          receive X-DCC headers.

     -h homedir
          overrides the default DCC home directory, which is often /var/dcc.

     -p protocol:filename | protocol:port@host
          specifies the protocol and address by which sendmail will contact
          dccm.  The default is a UNIX domain socket in the "run" directory,
          often /var/run/dcc/dccm.  (See also -R) This protocol and address
          must match the value in sendmail.cf.  This mechanism can be used to
          connect dccm on one computer to sendmail on another computer when a
          port and host name or IP address are used.

     -m map
          specifies a name or path of the memory mapped parameter file instead
          of the default map file in the DCC home directory.  It should be
          created with the cdcc(8) command.

     -w whiteclnt
          specifies an optional file containing SMTP client IP addresses, SMTP
          envelope values, and header values of mail that is spam or is not
          spam and does not need a X-DCC header, and whose checksums should
          not be reported to the DCC server.

          If the pathname whiteclnt is not absolute, it is relative to the DCC
          home directory.  The format of the dccm whiteclnt file is the same
          as the whitelist files used by dbclean(8) and the whiteclnt file
          used by dccproc(8).  See dcc(8) for a description of DCC white and
          blacklists.  Because the contents of the whiteclnt file are used
          frequently, a companion file is automatically created and main-
          tained.  It has the same pathname but with an added suffix of .dccw
          and contains a memory mapped hash table of the main file.

          A white-list entry ("OK") or two or more semi-white-listings ("OK2")
          for the message's checksums prevents all of the message's checksums
          from being reported to the DCC server and the addition of a X-DCC
          header line by dccm (except for env_To checksums or when -W is
          used).  A white-listing entry for a checksum also prevents rejecting
          or discarding the message based on DCC recipient counts as specified
          by -a and -t.  Otherwise, one or more checksums with blacklisting
          entries ("MANY") cause all of the message's checksums to be reported
          to the server with an addressee count of "MANY".

          White-list env_To values are handy for white-listing or exempting
          destination addresses such as Postmaster from filtering and for mak-
          ing "spam traps" of addresses that should never receive mail.  First
          an entry for the official envelope Rcpt To value is sought.  If that
          is not found, dccm looks for an entry for the sendmail "user"
          string.  Mail sent to blacklisted addresses or with other black-
          listed values such as From or env_From values is reported to the DCC
          server as spam or with target counts of millions.

          If the message has a single recipient, an env_To whiteclnt entry of
          "OK" for the checksum of its recipient address acts like any other
          whiteclnt entry of "OK."  When the SMTP message has more than one
          recipient, the effects can be complicated.  When a message has sev-
          eral recipients with some but not all listed in the whiteclnt file,
          dccm tries comply with the wishes of the users who want filtering as
          well as those who don't by silently not delivering the message to
          those who want filtering (i.e. are not white-listed) and delivering
          the message to don't want filtering.

          Consider -W or a option dcc-off line in whitelist files to turn off
          DCC filtering.

     -U userdirs
          enables private whitelist and log files.  Each target of a message
          can have a directory of log files named usedirs/${dcc_userdir}/log
          where ${dcc_userdir} is the sendmail.cf macro described below.  If
          ${dcc_userdir} is not set, userdirs/${rcpt_mailer}/${rcpt_addr}/log
          is used.  If it is not absolute, userdirs is relative to the DCC
          home directory.  The sub-directory prefixes for -l logdir are not
          honored.  The directory containing the log files must be named log
          and it must be writable by the dccm process.  Each log directory
          must exist or logging for the corresponding is silently disabled.
          The files created in the log directory are owned by the UID of the
          dccm process, but they have group and other read and write permis-
          sions copied from the corresponding log directory.  To ensure the
          privacy of mail, it may be good to make the directories readable
          only by owner and group, and to use a cron script that changes the
          owner of each file to match the grandparent addr directory.

          There can also be userdirs/${dcc_userdir}/whiteclnt, or if
          ${dcc_userdir} is not set, userdirs/${rcpt_mailer}/${rcpt_addr} per-
          user whitelist files.  The name of each file must be whiteclnt.
          Every checksum including the env_to and sendmail "user" values are
          looked for first in the userdirs/mailer/addr/whiteclnt and list then
          in the global -w whiteclnt list.  A missing per-address whiteclnt
          file is the same as an empty file.  Relative paths for whitelists
          included in per-address whiteclnt are resolved in the DCC home
          directory.  The whiteclnt files and the addr directories containing
          them must be writable by the dccm process.

          The most likely value of mailer is local.  Appropriate values for
          both mailer and addr can be seen by examining env_To lines in -l
          logdir files.

     -a IGNORE | REJECT | DISCARD
          specifies the action taken when DCC server counts or -t thresholds
          say that a message is unsolicited bulk.  IGNORE causes the message
          to be unaffected except for adding the X-DCC header line to the mes-
          sage.  This turns off DCC filtering.

          Spam can also be REJECTed, or accepted and silently DISCARDed with-
          out being delivered to local mailboxes.  The default is REJECT.

          With an action of REJECT or DISCARD, spam sent to both white-listed
          targets and non-white-listed targets is delivered to white-listed
          targets and if possible, silently discarded for non-white-listed
          targets.  This is not possible if there are too many non-white-
          listed targets to be saved in a buffer of about 500 bytes.

          Determinations that mail is or is not spam from sendmail via
          ${dcc_isspam} or ${dcc_notspam} macros override -a.  The effects of
          the -w whiteclnt are also not affected by -a.

     -t type,[log-thold,]rej-thold
          sets logging and "spam" thresholds for checksum type.  The checksum
          types are IP, env_From, From, Message-ID, Received, Body, Fuz1, and
          Fuz2.  The string ALL sets thresholds for all types, but is unlikely
          to be useful except for setting logging thresholds.  The string CMN
          specifies the commonly used checksums Body, Fuz1, and Fuz2.
          Rej-thold and log-thold must be numbers, the string NEVER, or the
          string MANY indicating millions of targets.  Counts from the DCC
          server as large as the threshold for any single type are taken as
          sufficient evidence that the message should be logged or rejected.

          Log-thold is the threshold at which messages are logged.  It can be
          handy to log messages at a lower threshold to find solicited bulk
          mail sources such as mailing lists.  If no logging threshold is set,
          only rejected mail and messages with complicated combinations of
          white and blacklisting are logged.  Messages that reach at least one
          of their rejection thresholds are logged regardless of logging
          thresholds.

          Rej-thold is the threshold at which messages are considered "bulk,"
          and so should be rejected or discard if not white-listed.  Use -a
          REJECT or -a Discard to reject or discard bulk mail that is not
          white-listed.  Use -a IGNORE to only add X-DCC headers with the
          "bulk" string.

          The checksums of locally white-listed messages are not checked with
          the DCC server and so only the number of targets of the current
          instance of a white-listed message are compared against the thresh-
          olds.

          The default is -t ALL,NEVER, so that nothing is discarded or logged.
          A common choice is -t CMN,25,50 to reject or discard mail with com-
          mon bodies except as overridden by the whitelist of the DCC server,
          the sendmail ${dcc_isspam} and ${dcc_notspam} macros, and -g, and
          -w.

     -g [not-]type
          indicates that white-listed, OK or OK2, counts from the DCC server
          for a type of checksum are to be believed.  They should be ignored
          if prefixed with not-.  Type is one of the same set of strings as
          for -t.  Only IP, env_From, and From are likely choices.  By default
          all three are honored, and hence the need for not-.

     -S hdr
          adds to the list of substitute or locally chosen headers that are
          checked with the -w whiteclnt file and sent to the DCC server.  The
          checksum of the last header of type hdr found in the message is
          checked.  Hdr can be HELO to specify the SMTP envelope HELO value.
          Hdr can also be mail_host to specify the sendmail "resolved" host
          name from the Mail_from value in the SMTP envelope.  As many as 6
          different substitute headers can be specified, but only the checksum
          of the first of the six will be sent to the DCC server.

     -l logdir
          specifies a directory in which files containing copies of messages
          processed by dccm are kept.  All messages logged are copied to the
          -l logdir directory.  They can also be copied to per-user directo-
          ries specified with -U.  Information about other recipients of a
          message is deleted from the per-user copies.

          If logdir starts with D?, log files are put into subdirectories of
          the form logdir/JJJ where JJJ is the current julian day.  H?logdir
          puts logs files into subdirectories of the form logdir/JJJ/HH where
          HH is the current hour.  M?logdir puts log files into subdirectories
          of the form logdir/JJJ/HH/MM where MM is the current minute.  See
          the FILES section below concerning the contents of the files.

          The directory is relative to the DCC home directory if it is not
          absolute

     -R rundir
          specifies the "run" directory where the UNIX domain socket and file
          containing the daemon's process ID are stored.  The default value is
          often /var/run/dcc.

     -r rejection-msg
          specifies the rejection message for unsolicited bulk mail or for
          mail temporarily blocked by greylisting when -G is specified.  The
          first rejection-msg replaces the default bulk mail rejection mes-
          sage, "5.7.1 550 mail %s from %s rejected by DCC".  The second
          replaces "4.2.1 452 mail %s from %s greylist temporary embargoed".
          There can be zero, one, or two "%s" strings.  The first is replaced
          by the sendmail queue ID and the second is replaced by the IP
          address of the SMTP client.

          A common alternate for the bulk mail rejection message is "4.7.1 451
          Access denied by DCC" to tell the sender to continue trying.  Use a
          4yz response with caution, because it is likely to delay for days a
          delivery failure message for false positives.  If the bulk mail
          rejection message does not start with a recognized error type and
          number, type 5.7.1 and 550 or 4.2.1 and 452 are used.

     -j maxjobs
          limits the number of simultaneous requests from sendmail that will
          be processed.  The default value is the maximum number that seems to
          be possible given the number of open files, select() bit masks, and
          so forth that are available.

     -B dnsbl-option
          enables DNS blacklist checks of the SMTP client IP address, SMTP
          envelope Mail_From sender domain name, and of host names in URLs in
          the message body.  Body URL blacklisting has far too many false pos-
          itives to use on abuse mailboxes.  It is less effective than
          greylisting with dccm(8) or dccifd(8) but can be useful in situa-
          tions where greylisting cannot be used.

          Dnsbl-option is either of the form set:option or of the form
          domain[,IPaddr[,bltype]].  Domain is a DNS blacklist domain such as
          example.com that will be searched.  IPaddr is the IP address in the
          DNS blacklist that indicates that the mail message is spam.
          127.0.0.1 is assumed if IPaddr is absent.  IPv6 addresses can be
          specified with the usual colon (:) notation.  Names can be used
          instead of numeric addresses.  The type of DNS blacklist is speci-
          fied by bltype as name, IPv4, or IPv6.  Given an envelope sender
          domain name or a domain name in a URL of spam.domain.org and a
          blacklist of type name, spam.domain.org.example.com will be tried.
          Blacklist types of IPv4 and IPv6 require that the domain name in a
          URL be resolved into an IPv4 or IPv6 address.  The address is then
          written as a reversed string of decimal octets to check the DNS
          blacklist, as in 2.0.0.127.example.com,

          More than one blacklist can be specified.  They are searched in
          order.  All searching is stopped at the first positive result.  Pos-
          itive results are ignored after being logged unless an option
          DNSBL-on line appears in the global or per-user whiteclnt file.

          -B set:debug sends more messages about all DNS resolutions to the
          system log.

          -B set:msg-secs=S limits dccm to S seconds total for checking all
          DNS blacklists.  The default is 20.

          -B set:URL-secs=S limits dccm to at most S seconds resolving and
          checking any single URL.  The default is 5.  Some spam contains
          dozens of URLs and that some "spamvertised" URLs contain host names
          that need minutes to resolve.  Busy mail systems cannot afford to
          spend minutes checking each incoming mail message.  In order to use
          typical single-threaded DNS resolver libraries, dccm(8) and
          dccifd(8) use fleets of helper processes.

          -B set:no-envelope says that SMTP client IP addresses and sender
          Mail_From domain names should not be checked in the following black-
          lists.  -B set:envelope restores the default for subsequently named
          blacklists.

          -B set:no-body says that URLs in the message body should not be
          checked in the in the following blacklists.  -B set:body restores
          the default for later blacklists.

          -B set:no-MX says MX servers of sender Mail_From domain names and
          host names in URLs should not be checked in the following black-
          lists.  -B set:MX restores the default.

     -L ltype,facility.level
          specifies how messages should be logged.  Ltype must be error or
          info to indicate which of the two types of messages are being con-
          trolled.  Level must be a syslog(3) level among EMERG, ALERT, CRIT,
          ERR, WARNING, NOTICE, INFO, and DEBUG.  Facility must be among AUTH,
          AUTHPRIV, CRON, DAEMON, FTP, KERN, LPR, MAIL, NEWS, USER, UUCP, and
          LOCAL0 through LOCAL7.  The default is equivalent to

                -L info,MAIL.NOTICE -L error,MAIL.ERR

     dccm normally sends counts of mail rejected and so forth the system log
     at midnight.  The SIGUSR1 signal sends an immediate report to the system
     log.  They will be repeated every 24 hours instead of at midnight.

SENDMAIL MACROS
     Sendmail can affect dccm with the values of some sendmail.cf macros.
     These macro names must be added to the Milter.macros option statements in
     sendmail.cf as in the example "Feature" file dcc.m4.

     ${dcc_isspam}  causes a mail message to be reported to the DCC server as
                    having been addressed to "MANY" recipients.  The
                    ${dcc_isspam} macro is ignored if the ${dcc_notspam} macro
                    is set to a non-null string

                    If the value of the ${dcc_isspam} is null, dccm uses SMTP
                    rejection messages controlled by -a and -r.  If the value
                    of the ${dcc_isspam} macro starts with "DISCARD", the mail
                    message is silently discarded as with -a DISCARD. This can
                    be handy for keeping "spammers" from knowing they are
                    sending to "spam traps."  If value of the macro not null
                    and does not start with "DISCARD", it is used as the SMTP
                    error message given to the SMTP client trying to send the
                    rejected message.  The message starts with an optional
                    SMTP error type and number followed by text.

                    The -a option does not effect messages marked spam with
                    ${dcc_isspam}.  When the ${dcc_isspam} macro is set, the
                    message is rejected or discarded despite local or DCC
                    database white-list entries.  The local white-list does
                    control whether the message's checksums will be reported
                    to the DCC server and an X-DCC SMTP header line will be
                    added.

     ${dcc_notspam}
                    causes a message not be considered unsolicited bulk
                    despite evidence to the contrary.  It also prevents dccm
                    from reporting the checksums of the message to the DCC
                    server and from adding an X-DCC header line.

                    When the macro is set by the sendmail.cf rules,
                    ${dcc_notspam} macros overrides DCC threshlds that say the
                    message should be rejected as well as the effects of the
                    ${dcc_isspam} macro.

     ${dcc_mail_host}
                    specifies the name of the SMTP client that is sending the
                    message.  This macro is usually the same as the mail_host
                    macro.  They can differ when a sendmail "smart relay" is
                    involved.  The ${dcc_mail_host} macro does not work if
                    FEATURE(delay_checks) is used.

     ${dcc_userdir}
                    is the per-user whitelist and log directory for a recipi-
                    ent.  If the macro is not set in sendmail.cf,
                    $&{rcpt_mailer}/$&{rcpt_addr} is assumed,but with the
                    recipient address converted to lower case.  Whatever value
                    is used, the directory name after the last slash (/) char-
                    acter is converted to lower case.  Any value containing
                    the string "/../" is ignored.

                    This macro also does not work if FEATURE(delay_checks) is
                    used.

                    The following two lines in a sendmail mc file have the
                    same effect as not defining the ${dcc_userdir} macro, pro-
                    vided FEATURE(dcc) is also used and the sendmail
                    cf/feature directory has a symbolic link to the
                    misc/dcc.m4 file.
     SLocal_check_rcpt
     R$*     $: $1 $(macro {dcc_userdir} $@ $&{rcpt_mailer}/$&{rcpt_addr} $))

FILES
     /var/dcc   is the DCC home directory in which other files are found.
     libexec/start-dccm
                is a script often used to the daemon.
     dcc/dcc_conf
                contains parameters used by the scripts to start DCC daemons
                and cron jobs.
     logdir     is an optional directory specified with -l and containing
                marked mail.  Each file in the directory contains one message,
                at least one of whose checksums reached its -t thresholds or
                that is interesting for some other reason.  Each file starts
                with lines containing the date when the message was received,
                the IP address of the SMTP client, and SMTP envelope values.
                Those lines are followed by the body of the SMTP message
                including its header as it was received by sendmail and with-
                out any new or changed header lines.  Only approximately the
                first 32 KBytes of the body are recorded unless modified by
                ./configure --with-max-log-size=xx The checksums for the mes-
                sage follow the body.  They are followed by lines indicating
                that the ${dcc_isspam} or ${dcc_notspam} sendmail.cf macros
                were set or one of the checksums is white- or blacklisted by
                the -w whiteclnt file.  Each file ends with the X-DCC header
                line added to the message and the disposition of the message
                including SMTP status message if appropriate.
     map        is the memory mapped file of information concerning DCC
                servers in the DCC home directory.
     whiteclnt  contains the client whitelist in the format described in
                dcc(8).
     whiteclnt.dccw
                is a memory mapped hash table of the whiteclnt file.
     dccm.pid   in the -R rundir directory contains daemon's process ID.  The
                string ``dccm'' is replaced by the file name containing the
                daemon to facilitate running multiple daemons, probably con-
                nected to remote instances of sendmail using TCP/IP instead of
                a UNIX domain socket.  See also -R.
     /var/run/dcc/dccm
                is the default UNIX domain socket used by the sendmail milter
                interface.  See also -R.
     sendmail.cf
                is the sendmail(8) control file.
     misc/dcc.m4
                sendmail mc file that should have a symbolic link in the send-
                mail cf/feature directory so that FEATURE(dcc) can be used in
                a sendmail mc file.

EXAMPLES
     Dccm should be started before sendmail with something like the script
     libexec/start-dccm.  It looks for common DCC parameters in the dcc_conf
     file in the DCC home directory.

     Those numbers should modified to fit local conditions.  It might be wise
     to replace the "100" numbers with much larger values or with "MANY" until
     a few weeks of monitoring the log directory show that sources of mailing
     lists are in the server's whitelist file (see dccd(8)) or the local
     whiteclnt file.

     It is usually necessary to regularly delete old log files with a script
     like libexec/cron-dccd.

     Sendmail must be built with the milter interface, such as by creating a
     devtools/Site/site.config.m4 or similar file containing something like
     the following lines:

           APPENDDEF(`conf_sendmail_ENVDEF', `-D_FFR_MILTER=1')
           APPENDDEF(`conf_libmilter_ENVDEF', `-D_FFR_MILTER=1')

     Appropriate lines invoking the milter interface must be added to
     sendmail.cf. It should be sufficient to copy the dcc.m4 file to the send-
     mail 8.11 cf/feature directory and add the line

           FEATURE(dcc)

     to the local .mc file.

SEE ALSO
     cdcc(8), dbclean(8), dcc(8), dccd(8), dblist(8), dccifd(8), dccproc(8),
     dccsight(8), sendmail(8).

HISTORY
     Implementation of dccm was started at Rhyolite Software in 2000.  This
     describes version 1.2.74.

BUGS
     dccm uses -t where dccproc(8) uses -c.

     On many systems with sendmail 8.11.3 and preceding, a bug in the sendmail
     milter mechanism causes dccm to die with a core file when given a signal.

FreeBSD 4.9                     March 20, 2005                     FreeBSD 4.9
