Bounce Handler
- Name
- User Guide
- Description
- Obtaining a Copy of the Plugin
- Requirements
- Recommended
- Installation
- Lightning Configuration/Installation Instructions
- Screencasts
- Configuration
- Configuring the Cronjob to Automatically Run Bounce Handler
- Command Line Interface
- Plugin Configs
- More on Scores, Thresholds, etc
- Special Debugging Mode
- FAQs
- Introduction to Bounce Handler Rules
- COPYRIGHT
Name
Bounce Handler for Dada Mail
User Guide
For a guide on using Bounce Handler, see the Dada Mail Manual:
http://dadamailproject.com/pro_dada/using_bounce_handler.html
For more information on Pro Dada/Dada Mail Manual:
http://dadamailproject.com/purchase/pro.html
Description
Bounce Handler intelligently handles bounces from Dada Mail list messages.
Messages sent to subscribers of your mailing list that bounce back will be directed to the List Administrator Address. This email account is then checked periodically by Bounce Handler.
Bounce Handler then reads awaiting messages and parses the messages in an attempt to understand why the message has bounced.
The parsed message will then be examined and an action will be taken.
The usual action that is taken is to apply a score to the email address that has bounced the message. Once the Score Threshold is reached, the email address is unsubscribed from the mailing list.
Obtaining a Copy of the Plugin
Bounce Handler is located in the, dada/plugins directory of the main Dada Mail distribution, under the name, bounce_handler.cgi
Requirements
Please make sure you have these requirements before installing this plugin:
A POP3 Email Account
Bounce Handler works by checking a email address via POP3. (IMAP is currently not supported).
You will need to create a new email address account for Bounce Handler to utilize.
Example: create bounces@yourdomain.com, where, yourdomain.com is the name of the domain Dada Mail is installed on.
Guidelines on this address:
Do NOT use this address for anything except Bounce Handler
No one will be checking this POP3 account via a mail reader.
Doing so won't break Dada Mail, but it will stop Bounce Handler from working correctly, if when checking messages, your mail reader then removes those messages from the POP3 account. If you do need to periodically check this inbox, make sure to have your mail reader set to not automatically remove the messages.
The email address MUST belong to the domain you have Dada Mail installed
Meaning, if your domain is, "yourdomain.com", the bounce email address should be something like, "bounces@yourdomain.com". In other words, do not use a Yahoo! Gmail, or Hotmail account for your bounce address. This will most likely disrupt all regular mail sending in Dada Mail.
Bounce Handler MUST be able to check the POP3 account
Check to make sure that the POP3 server (usually, port 110) is not blocked from requests coming from your hosting account server.
Recommended
These points are not required, but recommended to use Bounce Handler:
Ability to set Cron Jobs.
Bounce Handler can be configured to run automatically by using a cronjob.
If you do not know how to set up a cronjob, attempting to set one up for Dada Mail will result in much aggravation. Please read up on the topic before attempting!
Installation
This plugin can be installed during a Dada Mail install/upgrade, using the included installer that comes with Dada Mail. The below installation instructions go through how to install the plugin manually. We suggest using the Dada Mail Installer.
If you do install this way, note that you still have to create the create the Bounce Handler email account as well set the cronjob. Both are covered below.
Lightning Configuration/Installation Instructions
Create the bounce handler email account
Configure your mailing list to use this email address for its, "List Administrator Address"
Set this email address as our "List Administrator Address" in the list control panel, under,
Your Mailing List - List Information
Configure the plugin in your .dada_config file
(covered below)
chmod 755 the bounce_handler.cgi script
visit the plugin via a web browser.
Set the cronjob (optional)
Screencasts
Part 1
https://www.youtube.com/watch?v=tvdIj1s19Vo
Part 2
https://www.youtube.com/watch?v=CnsM994xa7A
Configuration
There's a few things you need to configure for this plugin. Configure the plugin variables in your .dada_config
file (not in the plugin itself!)
POP3 Server Information.
Create a new POP3 email account. This email account will be the address that bounced messages will be delivered to.
Open up your .dada_config
file for editing.
Search and see if the following lines are present:
# start cut for plugin configs
=cut
=cut
# end cut for plugin configs
If they are present, remove them.
You can then configure the plugin variables on these lines:
Bounce_Handler => {
Server => undef,
Username => undef,
Password => undef,
# etc.
},
For example:
Bounce_Handler => {
Server => 'mail.yourdomain.com',
Username => 'bounces+yourdomain.com',
Password => 'password',
# etc.
},
Set this address as the default List Administration Email Address
You may also want to set a default value for the, List Adminstration Email, so that all new lists already have Bounce Handler enabled.
Find this chunk of lines in your .dada_config
file:
# start cut for list settings defaults
=cut
%LIST_SETUP_INCLUDE = (
set_smtp_sender => 1, # For SMTP
add_sendmail_f_flag => 1, # For Sendmail Command
admin_email => 'bounces@example.com',
);
=cut
# end cut for list settings defaults
Remove the =cut lines, similar to before:
# start cut for list settings defaults
=cut
=cut
# end cut for list settings defaults
And then change the, admin_email
to your bounce handler email address:
%LIST_SETUP_INCLUDE = (
set_smtp_sender => 1, # For SMTP
add_sendmail_f_flag => 1, # For Sendmail Command
admin_email => 'bounces@yourdomain.com',
);
List Control Panel Menu
Now, edit your .dada_config
file, so that it shows Bounce Handler in the left-hand menu, under the, Plugins heading:
First, see if the following lines are present in your .dada_config
file:
# start cut for list control panel menu
=cut
=cut
# end cut for list control panel menu
If they are, remove them.
Then, find these lines:
# {-Title => 'Bounce Handler',
# -Title_URL => $PLUGIN_URL."/bounce_handler.cgi",
# -Function => 'bounce_handler',
# -Activated => 1,
# },
Uncomment the lines, by taking off the, "#"'s:
{-Title => 'Bounce Handler',
-Title_URL => $PLUGIN_URL."/bounce_handler.cgi",
-Function => 'bounce_handler',
-Activated => 1,
},
Save your .dada_config
file.
Telling Dada Mail to use Bounce Handler.
You're going to have to tell Dada Mail explicitly that you want bounces to go to the bounce handler. The first step is to set the Dada List Administrator to your bounce email address. You'll set this per mailing list in the each mailing list's control panel, under
Your Mailing List - List Information
After that, you'll need to configure outgoing email messages to set the Dada List Administrator address in the Return-Path
header. Sounds scary, but it's easy enough.
If you're using th sendmail command:
In the list control panel, go to Mail Sending - Sending Options and check: Add the Sendmail '-f' flag when sending messages ...
This should set the sending to the admin email, and in turn, set the Return-Path header. Dada Mail is shipped to have this option set by default.
If you're using SMTP sending:
In the list control panel, go to: Mail Sending - Sending Options and check the box labeled: Set the Sender of SMTP mailings to the list administration email address Dada Mail is shipped to have this option set by default.
If you're using Amazon SES:
Dada Mail will automatically have bounces go to the List Administration Address when using Amazon SES. Make sure before you configure any of the above that you Verify this email address, just like you would a Amazon SES Sender. If you do not, your messages sent through Dada Mail will not be delivered.
Testing
To test out any of these configurations, Send yourself a test message through Dada Mail and view the source of the message itself, in your mail reader. In the mail headers, you should see the Return-Path header:
Return-Path: <dadabounce@myhost.com>
Delivered-To: justin@myhost.com
Received: (qmail 75721 invoked from network); 12 May 2003 04:50:01 -0000
Received: from myhost.com (208.10.44.140)
by hedwig.myhost.com with SMTP; 12 May 2003 04:50:01 -0000
Date:Sun, 11 May 2003 23:50:01 -0500
From:justin <justin@myhost.com>
Subject:Test, Test, Test
To:justin@myhost.com
Sender:dadabounce@myhost.com
Reply-To:justin <justin@myhost.com>
Precedence:list
Content-type:text/plain; charset=iso-8859-1
The first line has the Return-Path header should have the Bounce Handler Email set:
Return-Path: <dadabounce@myhost.com>
In this example, my List Owner address, justin@myhost.com still occupies the To:
and Reply-To headers
, so whoever replies to my message will reply to me, not Bounce Handler.
Configuring the Cronjob to Automatically Run Bounce Handler
We're going to assume that you already know how to set up the actual cronjob, but we'll be explaining in depth on what the cronjob you need to set is.
Setting the cronjob via curl
Generally, setting the cronjob to have Bounce Handler run automatically, just means that you have to have a cronjob access a specific URL (via a utility, like curl). The URL looks something like this:
http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&verbose=1
Where, http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi is the URL to your copy of bounce_handler.cgi
You'll see the specific URL used for your installation of Dada Mail in the web-based control panel for Bounce Handler, under the fieldset legend, Manually Run Bounce Handler, under the heading, Manual Run URL:
This will have Bounce Handler check any awaiting messages.
A Pretty Good Guess of what the entire cronjob should be set to is located in the web-based crontrol panel for Bounce Handler, under the fieldset legend, Manually Run Bounce Handler, under the heading, curl command example (for a cronjob):
Customizing your cronjob with added parameters
passcode
Since anyone (or anything) can run your Bounce Handler, by following that same URL, (http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&verbose=1
), you can set up a simple Passcode, to have some semblence of security over who runs the program.
Set a passcode in Bounce Handler's Config variable, Manual_Run_Passcode. This is done in your .dada_config
file - the same place the mail server, username and password were set. Find the lines in your .dada_config
file that look like this:
$PLUGIN_CONFIGS = {
Bounce_Handler => {
Server => 'mail.yourdomain.com',
Username => 'bounces+yourdomain.com',
Password => 'password',
Port => undef,
USESSL => undef,
AUTH_MODE => undef,
Plugin_Name => undef,
Plugin_URL => undef,
Allow_Manual_Run => undef,
Manual_Run_Passcode => undef,
Enable_POP3_File_Locking => undef,
Log => undef,
MessagesAtOnce => undef,
Max_Size_Of_Any_Message => undef,
Rules => undef,
},
Find the config parameter named, Manual_Run_Passcode and set it to whatever you'd like this Passcode to be:
Manual_Run_Passcode => 'sneaky',
Then change the URL to include this passcode. In our examples, it would then look like this:
http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&passcode=sneaky
The example cronjob for curl in Bounce Handler's list control panel should also update use the new passcode.
messages
Sets how many messages should be checked and parsed in one execution of the program. Example:
http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?run=1&messages=10
verbose
When set to, 1, you'll receive the a report of how Bounce Handler is doing parsing and adding scores (and what not). This is sometimes not so desired, especially in a cron environment, since all this informaiton will be emailed to you (or someone) everytime the script is run.
If you set verbose to, "0", under normal operation, Bounce Handler won't show any output, but if there's a server error, you'll receive an email about it. This is probably a good thing. Example (for cronjob-run curl command):
* * * * * /usr/local/bin/curl -s --get --data run=1\;verbose=0 --url http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi
test
Runs Bounce Handler in test mode by checking the bounces and parsing them, but not actually carrying out any rules (no scores added, no email addresses unsubscribed).
Command Line Interface
There's a slew of optional arguments you can give to this script. To use Bounce Handler via the command line, first change into the directory that Bounce Handler resides in, and issue the command:
./bounce_handler.cgi --help
For a full list of parameters.
You may set the cronjob via the command line interface, rather than the web-based way. You may run into file permission problems when running it this way, depending on your server setup.
Command Line Interface for Cronjobs:
The secret is to actually have two commands in one. The first command changes into the same directory as the bounce_handler.cgi
script, the second invokes the script with the parameters you'd like.
For example:
*/5 * * * * cd /home/myaccount/cgi-bin/dada/plugins; /usr/bin/perl ./bounce_handler.cgi >/dev/null 2>&1
Where, /home/myaccount/cgi-bin/dada/plugins is the full path to the directory the dada_bounc_handler.pl
script resides.
Plugin Configs
These plugin configs are located in your .dada_config
file, as mentioned above.
Server, Username, Password
These configs need to be set, to hook Bounce Handler to the email address you want the bounce messages to go.
Port
Defaults to: 110 for regular connections, 995
for SSL connections.
Sets the port Bounce Handler uses connect to the POP server.
USESSL
Defaults to: 0. Set to, 1, if you'd like (and can) connect to the POP server with an SSL connection.
AUTH_MODE
Defaults to: BEST
Allowed parameters: BEST, PASS, APOP, CRAM-MD5.
If the default of, BEST isn't working, try the various allowed parameters. PASS passes the POP3 password in cleartext.
BEST
Tries, APOP, CRAM-MD5 and PASS modes, in that order of availability.
Plugin_Name
The name of this plugin.
Defaults to: Bounce Handler
Plugin_URL
The URL of the plugin. This is usually figured out by default, but if it's not (you'll know, as links are broken in the plugin and nothing seems to work) you may have to set this, manually.
Allow_Manual_Run
Defaults to 1 (enabled)
Sets whether you may use the manual run URL to run Bounce Handler. The manual run URL is what the curl-powered cronjob uses. If you want to disable this method, set this config variable to, 0
Manual_Run_Passcode
This is covered above, under, passcode
Enable_POP3_File_Locking
Defaults to 1
When enabled, Bounce Handler will use a lock file to make sure only one connection to the POP server is done at one time. Disable this by setting this parameter to, 0.
Log
Sets the path to the logfile Bounce Handler creates. Defaults to: bounces.txt
in your .dada_files/.logs
directory.
MessagesAtOnce
Sets how many messages are checke, per running of the plugin.
Defaults to: 100
Since there could be many bounce messages awaiting to be checked, there is a limit that's set on how many mesages are looked at, at one time. This also means that it may take a few runnings of the plugin to clear all the awaiting messages.
Max_Size_Of_Any_Message
Defaults to: 2,621,440 bytes (2.5 megs). Set in, octets
Sets the maximum size of any bounced message that Bounce Handler will deal with. Anything larger than this will simply be ignored and deleted.
More on Scores, Thresholds, etc
By default, Bounce Handler assigns a particular score to each email address that bounces back a message. These scores are tallied each time an email address bounces a message.
Since Dada Mail understands the differences between Hard Bounces and Soft Bounces, it'll append a smaller score for soft bounces, and a larger score for hard bounces. There's also a Decay Rate, an amount that all scores are decreased by, every time a mass mailing is sent out.
Once the email address's Bounce Score reaches the Threshold, the email address is then removed from the list.
Special Debugging Mode
manually_enter_bounces
Visit the plugin with the following query string,
http://example.com/cgi-bin/dada/plugins/bounce_handler.cgi?flavor=manually_enter_bounces
(manually_enter_bounces
) to enter a special debugging mode, where you can just paste in the source of a bounce message, and see how Bounce Handler will handle it, without actually processing the message.
FAQs
Bounce Email Address
Do you use only one Bounce Email Address for all the mailing lists?
Yes.
Even though there's only one Bounce Email Address, it is used by all the mailing lists of your Dada Mail, but Bounce Handler will work with every mailing list individually. Each mailing list also has a separate Bounce Scorecard.
Introduction to Bounce Handler Rules
Bounce Handler Rules are what's used when the bounced message is parsed and examined. They're a set of things to look for, and match, in an attempt to figure out what type of bounced message was given back.
An example Rule:
{
exim_user_unknown => {
Examine => {
Message_Fields => {
Status => [qw(5.x.y)],
Guessed_MTA => [qw(Exim)],
},
Data => {
Email => 'is_valid',
List => 'is_valid',
}
},
Action => {
add_to_score => 'hardbounce_score',
},
}
},
exim_user_unknown is the title of the rule - just a label, nothing else.
Examine holds a set of parameters that the handler looks at when trying to figure out what to do with a bounced message. This example has a Message_Fields entry and inside that, a Status entry. The Status entry holds a list of status codes. The ones in shown there all correspond to hard bounces; the mailbox probably doesn't exist.
Message_Fields also hold a, Guessed_MTA entry - it's explicitly looking for a bounce back from the, Exim mail server.
Examine also holds a Data entry, which holds the Email or List entries, or both. Their values are either 'is_valid', or 'is_invalid'.
So, to sum this all up, this rule will match a message that has Status: Message Field contaning a user unknown error code, (5.1.1, etc) and also a Guessed_MTA Message Field containing, Exim. The message also has to be parsed to have found a valid email and list name.
If this all matches, the Action is... acted upon. In this case, the offending email address will be appended a, Bounce Score of, whatever, hardbounce_score is, which by is default, 4.
If you would like to have the bounced address automatically removed, without any sort of scoring happening, change the action from,
add_to_score => 'hardbounce_score',
to:
unsubscribe_bounced_email => 'from_list'
Also, changing from_list, to from_all_lists will do the trick.
Here's a schematic of all the different things you can do:
{
rule_name => {
Examine => {
Message_Fields => {
Status => qw([ ]),
Last-Attempt-Date => qw([ ]),
Action => qw([ ]),
Status => qw([ ]),
Diagnostic-Code => qw([ ]),
Final-Recipient => qw([ ]),
Remote-MTA => qw([ ]),
# etc, etc, etc
},
Data => {
Email => 'is_valid' | 'is_invalid'
List => 'is_valid' | 'is_invalid'
}
},
Action => {
add_to_score => $x, # where, "$x" is a number
unsubscribe_bounced_email => 'from_list' | 'from_all_lists',
},
},
Rules also support the use of regular expressions for matching any of the Message_Fields.
To tell the parser that you're using a regular expression, make the Message_Field key end in '_regex':
'Final-Recipient_regex' => [(qr/RFC822/)],
Customizing Bounce Handler Rules
The default set of Bounce Handler Rules can be found at,
dada/data/bounce_handler_rules.pl
Bounce Handler will also look for a copy of this file in your
.dada_files/.configs
directory, which you can then customize with your own rules. When Dada Mail is upgrading, your customizations will then not be lost.
COPYRIGHT
Copyright (c) 1999 - 2014 Justin Simoni http://justinsimoni.com All rights reserved.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
Parts of this script were swiped from Mail::Bounce::Qmail module, fetched from here:
http://mikoto.sapporo.iij.ad.jp/cgi-bin/cvsweb.cgi/fmlsrc/fml/lib/Mail/Bounce/Qmail.pm
The copyright of that code stated:
Copyright (C) 2001,2002,2003 Ken'ichi Fukamachi All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Thanks Ken'ichi