Loading

Bounce Handler

Name

Bounce Handler For Dada Mail

User Guide

The below documentation goes into detail on how to install and configure Bounce Handler. A user guide for Bounce Handler is available in the Dada Mail Manual chapter, Using the Dada Bounce Handler:

http://dadamailproject.com/pro_dada/using_bounce_handler.html

Description

Bounce Handler intelligently handles bounces from Dada Mail list messages.

Messages sent to subscribers of your mailing list that bounce back will be directed to the List Administrator Address. This email account is then checked periodically by Bounce Handler.

Bounce Handler then reads awaiting messages and parses the messages in an attempt to understand why the message has bounced.

The parsed message will then be examined and an action will be taken.

The usual action that is taken is to apply a score to the email address that has bounced the message. Once the Score Threshold is reached, the email address is unsubscribed from the mailing list.

Obtaining a Copy of the Plugin

Bounce Handler is located in the, dada/plugins directory of the main Dada Mail distribution, under the name, bounce_handler.cgi

Requirements

Please make sure you have these requirements before installing this plugin:

Recommended

These points are not required, but recommended to use Bounce Handler:

Installation

This plugin can be installed during a Dada Mail install/upgrade, using the included installer that comes with Dada Mail. The below installation instructions go through how to install the plugin manually. We suggest using the Dada Mail Installer.

If you do install this way, note that you still have to create the create the Bounce Handler email account as well set the cronjob. Both are covered below.

Lightning Configuration/Installation Instructions

Screencasts

Part 1

Part 2

Configuration

There's a few things you need to configure for this plugin. Configure the plugin variables in your .dada_config file (not in the plugin itself!)

POP3 Server Information.

Create a new POP3 email account. This email account will be the address that bounced messages will be delivered to.

Open up your .dada_config file for editing.

Search and see if the following lines are present:

 # start cut for plugin configs
 =cut

 =cut
 # end cut for plugin configs

If they are present, remove them.

You can then configure the plugin variables on these lines:

        Bounce_Handler => { 
        
            Server                                      => undef, 
            Username                                    => undef, 
            Password                                    => undef, 
            # etc.
        },
        

For example:

Bounce_Handler => {

                Server                                          => 'mail.yourdomain.com', 
                Username                                        => 'bounces+yourdomain.com', 
                Password                                        => 'password', 
            # etc.
},

Set this address as the default List Administration Email Address

You may also want to set a default value for the, List Adminstration Email, so that all new lists already have Bounce Handler enabled.

Find this chunk of lines in your .dada_config file:

        # start cut for list settings defaults
        =cut
        
        %LIST_SETUP_INCLUDE = (
                set_smtp_sender              => 1, # For SMTP   
                add_sendmail_f_flag          => 1, # For Sendmail Command
                admin_email                  => 'bounces@example.com',
        );
        
        =cut
        # end cut for list settings defaults

Remove the =cut lines, similar to before:

        # start cut for list settings defaults
        =cut

        =cut
        # end cut for list settings defaults

And then change the, admin_email to your bounce handler email address:

        %LIST_SETUP_INCLUDE = (
                set_smtp_sender              => 1, # For SMTP   
                add_sendmail_f_flag          => 1, # For Sendmail Command
                admin_email                  => 'bounces@yourdomain.com',
        );

List Control Panel Menu

Now, edit your .dada_config file, so that it shows Bounce Handler in the left-hand menu, under the, Plugins heading:

First, see if the following lines are present in your .dada_config file:

 # start cut for list control panel menu
 =cut

 =cut
 # end cut for list control panel menu

If they are, remove them.

Then, find these lines:

 #                                      {-Title      => 'Bounce Handler',
 #                                       -Title_URL  => $PLUGIN_URL."/bounce_handler.cgi",
 #                                       -Function   => 'bounce_handler',
 #                                       -Activated  => 1,
 #                                      },

Uncomment the lines, by taking off the, "#"'s:

                                        {-Title      => 'Bounce Handler',
                                         -Title_URL  => $PLUGIN_URL."/bounce_handler.cgi",
                                         -Function   => 'bounce_handler',
                                         -Activated  => 1,
                                        },

Save your .dada_config file.

Telling Dada Mail to use Bounce Handler.

You're going to have to tell Dada Mail explicitly that you want bounces to go to the bounce handler. The first step is to set the Dada List Administrator to your bounce email address. You'll set this per mailing list in the each mailing list's control panel, under

Your Mailing List - Change List Information

After that, you'll need to configure outgoing email messages to set the Dada List Administrator address in the Return-Path header. Sounds scary, but it's easy enough.

If you're using th sendmail command:

In the list control panel, go to Mail Sending - Sending Options and check: Add the Sendmail '-f' flag when sending messages ...

This should set the sending to the admin email, and in turn, set the Return-Path header. Dada Mail is shipped to have this option set by default.

If you're using SMTP sending:

In the list control panel, go to: Mail Sending - Sending Options and check the box labeled: Set the Sender of SMTP mailings to the list administration email address Dada Mail is shipped to have this option set by default.

If you're using Amazon SES:

Dada Mail will automatically have bounces go to the List Administration Address when using Amazon SES. Make sure before you configure any of the above that you Verify this email address, just like you would a Amazon SES Sender. If you do not, your messages sent through Dada Mail will not be delivered.

Testing

To test out any of these configurations, Send yourself a test message through Dada Mail and view the source of the message itself, in your mail reader. In the mail headers, you should see the Return-Path header:

 Return-Path: <dadabounce@myhost.com>
 Delivered-To: justin@myhost.com
 Received: (qmail 75721 invoked from network); 12 May 2003 04:50:01 -0000
 Received: from myhost.com (208.10.44.140)
   by hedwig.myhost.com with SMTP; 12 May 2003 04:50:01 -0000
 Date:Sun, 11 May 2003 23:50:01 -0500
 From:justin <justin@myhost.com>
 Subject:Test, Test, Test
 To:justin@myhost.com
 Sender:dadabounce@myhost.com
 Reply-To:justin <justin@myhost.com>
 Precedence:list
 Content-type:text/plain; charset=iso-8859-1

The first line has the Return-Path header should have the Bounce Handler Email set:

        Return-Path: <dadabounce@myhost.com>

In this example, my List Owner address, justin@myhost.com still occupies the To: and Reply-To headers, so whoever replies to my message will reply to me, not Bounce Handler.

Configuring the Cronjob to Automatically Run Bounce Handler

We're going to assume that you already know how to set up the actual cronjob, but we'll be explaining in depth on what the cronjob you need to set is.

Setting the cronjob via curl

Generally, setting the cronjob to have Bounce Handler run automatically, just means that you have to have a cronjob access a specific URL (via a utility, like curl). The URL looks something like this:

 http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&verbose=1

Where, http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi is the URL to your copy of bounce_handler.cgi

You'll see the specific URL used for your installation of Dada Mail in the web-based control panel for Bounce Handler, under the fieldset legend, Manually Run Bounce Handler, under the heading, Manual Run URL:

This will have Bounce Handler check any awaiting messages.

A Pretty Good Guess of what the entire cronjob should be set to is located in the web-based crontrol panel for Bounce Handler, under the fieldset legend, Manually Run Bounce Handler, under the heading, curl command example (for a cronjob):

Customizing your cronjob with added parameters

passcode

Since anyone (or anything) can run your Bounce Handler, by following that same URL, (http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&verbose=1), you can set up a simple Passcode, to have some semblence of security over who runs the program.

Set a passcode in Bounce Handler's Config variable, Manual_Run_Passcode. This is done in your .dada_config file - the same place the mail server, username and password were set. Find the lines in your .dada_config file that look like this:

        $PLUGIN_CONFIGS = { 
        
                Bounce_Handler => {
                        Server                      => 'mail.yourdomain.com', 
                        Username                    => 'bounces+yourdomain.com', 
                        Password                    => 'password', 
                        Port                        => undef,
                        USESSL                      => undef,
                        AUTH_MODE                   => undef,
                        Plugin_Name                 => undef,
                        Plugin_URL                  => undef,
                        Allow_Manual_Run            => undef,
                        Manual_Run_Passcode         => undef,
                        Enable_POP3_File_Locking    => undef, 
                        Log                         => undef,
                        MessagesAtOnce              => undef,
                        Max_Size_Of_Any_Message     => undef,
                        Rules                       => undef,
                        
                },

Find the config parameter named, Manual_Run_Passcode and set it to whatever you'd like this Passcode to be:

                Manual_Run_Passcode         => 'sneaky',

Then change the URL to include this passcode. In our examples, it would then look like this:

 http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&passcode=sneaky

The example cronjob for curl in Bounce Handler's list control panel should also update use the new passcode.

messages

Sets how many messages should be checked and parsed in one execution of the program. Example:

 http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&messages=10

verbose

When set to, 1, you'll receive the a report of how Bounce Handler is doing parsing and adding scores (and what not). This is sometimes not so desired, especially in a cron environment, since all this informaiton will be emailed to you (or someone) everytime the script is run.

If you set verbose to, "0", under normal operation, Bounce Handler won't show any output, but if there's a server error, you'll receive an email about it. This is probably a good thing. Example (for cronjob-run curl command):

 * * * * * /usr/local/bin/curl -s --get --data run=1\;verbose=0 --url http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi

test

Runs Bounce Handler in test mode by checking the bounces and parsing them, but not actually carrying out any rules (no scores added, no email addresses unsubscribed).

Command Line Interface

There's a slew of optional arguments you can give to this script. To use Bounce Handler via the command line, first change into the directory that Bounce Handler resides in, and issue the command:

 ./bounce_handler.cgi --help

For a full list of parameters.

You may set the cronjob via the command line interface, rather than the web-based way. You may run into file permission problems when running it this way, depending on your server setup.

Command Line Interface for Cronjobs:

The secret is to actually have two commands in one. The first command changes into the same directory as the bounce_handler.cgi script, the second invokes the script with the parameters you'd like.

For example:

 */5 * * * * cd /home/myaccount/cgi-bin/dada/plugins; /usr/bin/perl ./bounce_handler.cgi  >/dev/null 2>&1

Where, /home/myaccount/cgi-bin/dada/plugins is the full path to the directory the dada_bounc_handler.pl script resides.

Plugin Configs

These plugin configs are located in your .dada_config file, as mentioned above.

Server, Username, Password

These configs need to be set, to hook Bounce Handler to the email address you want the bounce messages to go.

Port

Defaults to: 110 for regular connections, 995 for SSL connections.

Sets the port Bounce Handler uses connect to the POP server.

USESSL

Defaults to: 0. Set to, 1, if you'd like (and can) connect to the POP server with an SSL connection.

AUTH_MODE

Defaults to: BEST

Allowed parameters: BEST, PASS, APOP, CRAM-MD5.

If the default of, BEST isn't working, try the various allowed parameters. PASS passes the POP3 password in cleartext.

Plugin_Name

The name of this plugin.

Defaults to: Bounce Handler

Plugin_URL

The URL of the plugin. This is usually figured out by default, but if it's not (you'll know, as links are broken in the plugin and nothing seems to work) you may have to set this, manually.

Allow_Manual_Run

Defaults to 1 (enabled)

Sets whether you may use the manual run URL to run Bounce Handler. The manual run URL is what the curl-powered cronjob uses. If you want to disable this method, set this config variable to, 0

Manual_Run_Passcode

This is covered above, under, passcode

Enable_POP3_File_Locking

Defaults to 1

When enabled, Bounce Handler will use a lock file to make sure only one connection to the POP server is done at one time. Disable this by setting this parameter to, 0.

Log

Sets the path to the logfile Bounce Handler creates. Defaults to: bounces.txt in your .dada_files/.logs directory.

MessagesAtOnce

Sets how many messages are checke, per running of the plugin.

Defaults to: 100

Since there could be many bounce messages awaiting to be checked, there is a limit that's set on how many mesages are looked at, at one time. This also means that it may take a few runnings of the plugin to clear all the awaiting messages.

Max_Size_Of_Any_Message

Defaults to: 2,621,440 bytes (2.5 megs). Set in, octets

Sets the maximum size of any bounced message that Bounce Handler will deal with. Anything larger than this will simply be ignored and deleted.

More on Scores, Thresholds, etc

By default, Bounce Handler assigns a particular score to each email address that bounces back a message. These scores are tallied each time an email address bounces a message.

Since Dada Mail understands the differences between Hard Bounces and Soft Bounces, it'll append a smaller score for soft bounces, and a larger score for hard bounces. There's also a Decay Rate, an amount that all scores are decreased by, every time a mass mailing is sent out.

Once the email address's Bounce Score reaches the Threshold, the email address is then removed from the list.

FAQs

Bounce Email Address

Do you use only one Bounce Email Address for all the mailing lists?

Yes.

Even though there's only one Bounce Email Address, it is used by all the mailing lists of your Dada Mail, but Bounce Handler will work with every mailing list individually. Each mailing list also has a separate Bounce Scorecard.

Introduction to Bounce Handler Rules

Bounce Handler Rules are what's used when the bounced message is parsed and examined. They're a set of things to look for, and match, in an attempt to figure out what type of bounced message was given back.

An example Rule:

     {
        exim_user_unknown => { 
            Examine => { 
                Message_Fields => { 
                    Status      => [qw(5.x.y)], 
                    Guessed_MTA => [qw(Exim)],  
                }, 
                Data => { 
                    Email       => 'is_valid',
                    List        => 'is_valid', 
                }
            },
                Action => { 
                     add_to_score => 'hardbounce_score',
                }, 
            }
    }, 

exim_user_unknown is the title of the rule - just a label, nothing else.

Examine holds a set of parameters that the handler looks at when trying to figure out what to do with a bounced message. This example has a Message_Fields entry and inside that, a Status entry. The Status entry holds a list of status codes. The ones in shown there all correspond to hard bounces; the mailbox probably doesn't exist.

Message_Fields also hold a, Guessed_MTA entry - it's explicitly looking for a bounce back from the, Exim mail server.

Examine also holds a Data entry, which holds the Email or List entries, or both. Their values are either 'is_valid', or 'is_invalid'.

So, to sum this all up, this rule will match a message that has Status: Message Field contaning a user unknown error code, (5.1.1, etc) and also a Guessed_MTA Message Field containing, Exim. The message also has to be parsed to have found a valid email and list name.

If this all matches, the Action is... acted upon. In this case, the offending email address will be appended a, Bounce Score of, whatever, hardbounce_score is, which by is default, 4.

If you would like to have the bounced address automatically removed, without any sort of scoring happening, change the action from,

    add_to_score => 'hardbounce_score',

to:

    unsubscribe_bounced_email => 'from_list'

Also, changing from_list, to from_all_lists will do the trick.

Here's a schematic of all the different things you can do:

 {
 rule_name => {
         Examine => {
                Message_Fields => {
                        Status               => qw([    ]), 
                        Last-Attempt-Date    => qw([    ]), 
                        Action               => qw([    ]), 
                        Status               => qw([    ]), 
                        Diagnostic-Code      => qw([    ]), 
                        Final-Recipient      => qw([    ]), 
                        Remote-MTA           => qw([    ]), 
                        # etc, etc, etc
                        
                },
                Data => { 
                        Email => 'is_valid' | 'is_invalid' 
                        List  => 'is_valid' | 'is_invalid' 
                }
        },
        Action => { 
                   add_to_score             =>  $x, # where, "$x" is a number
                           unsubscribe_bounced_email => 'from_list' | 'from_all_lists',
        },
 },     

Rules also support the use of regular expressions for matching any of the Message_Fields.

To tell the parser that you're using a regular expression, make the Message_Field key end in '_regex':

 'Final-Recipient_regex' => [(qr/RFC822/)], 

Customizing Bounce Handler Rules

The default set of Bounce Handler Rules can be found at,

dada/data/bounce_handler_rules.pl

Bounce Handler will also look for a copy of this file in your

.dada_files/.configs

directory, which you can then customize with your own rules. When Dada Mail is upgrading, your customizations will then not be lost.

COPYRIGHT

Copyright (c) 1999 - 2014 Justin Simoni http://justinsimoni.com All rights reserved.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

Parts of this script were swiped from Mail::Bounce::Qmail module, fetched from here:

http://mikoto.sapporo.iij.ad.jp/cgi-bin/cvsweb.cgi/fmlsrc/fml/lib/Mail/Bounce/Qmail.pm

The copyright of that code stated:

Copyright (C) 2001,2002,2003 Ken'ichi Fukamachi All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

Thanks Ken'ichi


Dada Mail Project

Download

Installation

Support