Auto-Pickup of Mailings (Big update on next major version 2.11)

 
From: "Dada Mail" <dada@PROTECTED>
Date: August 17th 2006
Recently, I've been burning the midnight oil to work on a major problem with Dada Mail, a lack of the ability to detect when a mailing has been dropped and to pick up the dropped mailing, right where it left off and continue along its way.�

This is such a�prevalent problem, because of two issues:�

* el cheapo web hosts have strict limits on how many messages can be sent out in a window of time.�
�� �� �� �No problem in Dada Mail, you can just set the batch settings to compensate.�
� This leads to longer mail sendings, which leads to the second problem:�

* el cheapo web hosts will kill long-running processes, like Dada Mail's sending process.�

What happens if both these things are the case, is that your mailing will fail, somewhere before it's completed. Currently, there's no way to re-pickup a mailing exactly where it left off.�

The closest there is, is the form fields in the, "Advanced" "Send a List Message" screen, that allows, (heh), you to re-fill out the Subject, and Message Body form and then say where you think the mailing should restart at.�

Talk about inefficient and clumsy. Forget that if you have to do it more than once, you'll just give up on the sending and shake your fist at the air (or me, hey).�

So, I've been working on a solution to this problem, using basically what I wrote for here:�


Concerning this very problem.�

And, I'm very happy I've gotten really far in experimenting with a solution.�

Currently, this is how the solution works:�

When you click, "Submit my Message", a few things get created:�

* A copy of the message you're sending out
* A counter - to count how many messages have been sent (starts at 0, obv)
* A file that keeps track of the last time something was mailed,�
* A file that keeps the # of people that will receive the message�
* And, the actual temporary copy of the subscription list, with some meta-information

Every time a message is sent out, 1 is added to the counter. If everything works well, well that's it! Nothing to worry about.�

Let's say the mailing fails about halfway through a list of 100 subscribers at email # 50.�

With the auto-pickup enabled, it'll basically go through all the information above, find out where the message sending stopped, and tell Dada Mail to start at email # 51, using the temporary copy of the message. And that's it! Off to the races. This can happen as many times as the message sending is dropped.�

So, how do you monitor your mailing? That's another thing that's sort of lacking in Dada Mail. Currently, you can rifle through the logs, which isn't very fun, or, you have to rely on the batch emails to be sent, which can be spotty - as they only show the intervals that the batches represent.�

So, the solution I've made, is a little monitoring tool, that'll go through the same meta information we now save, and,�completely�independent of the sending process, will peek and see where you're at. Here's what it looks like (see the attachment if this doesn't come up)�


The screen that this is on, will refresh itself every 10 seconds or so. Every time it does, the numbers in the table will update to reflect where the sending process is.�

Now, it's pretty bulletproof - you can refresh the page yourself - no problem, you can navigate away from this screen, come back - you won't kill the sending process doing it - again, this is totally independent of the sending process - it doesn't control it directly, it just reports back the information the sending process itself is saving.�

Now, if something goes wrong in the sending process, you'll most likely get a screen that looks like this - I've turned off the auto-pickup feature. If it was on, you wouldn't even get this:

So, you get a little message saying, "Hey! Things have stopped!" And easy instructions on how to fix things (click a button). But, like I said, only if you turned, auto-pickup off will you even see this error.�


So I know this is a lot to take in, but to answer some questions that come up - here's a little "huh?" section,�

* Mailings should work almost exactly as before, so if this wasn't a problem before, don't think the mailing process will be much different for you. Only *if* you enable this auto-pickup feature, will you see much of a difference.�

* The auto-pickup works very lazily. Finding out if a mailing has dropped is actually fairly annoying - since there's no good, Pure-perl solution in seeing if a process is still running! It's harder than just looking at what the, "ps" command give back, unfortunetley.�

This is also complicated by the fact that when the mailing is dropped by the server, NO error message is recorded, the script is silently dropped. Which, is annoying.�

So you have to use basically second-hand information. For this current scheme, you look and see when the last mailing was, and if it was an incredibly long time, you say, "hey, I don't think anything is happening, let's kick the tires".�

I'm a *little* uneasy about this feature for this lazy behavior, but I'm not quite sure what to do about it (if there are any solutions, let me know!). Currently, the worst case scenario, is that you'll have some sort of doublings of the mail sending. Not what we want, but if the file locking on your server is working, this *might* not be a problem, except for a few minor race conditions.�



So, that's the long of it. Working with the feature should be a breeze: Enable the auto-pickup feature and leave your browser on the monitoring screen (or at least check in every once in a while). If you don't want to use the monitoring screen, you can use a small cron-runned script that'll do the monitoring for you. Again, the monitoring screen only reports information that's being logged by the sending process itself, it doesn't run the show, so if your web browser loses a connection from the server, no problem. The mailing, if dropped, will wait for you.�

My plan is to release this as I have it basically now alpha as version 2.11 on Monday, so anyone that wants to, can play with it. It's currently built with scotch tape and dental floss, so I'm not too worried about it breaking - it will, but maybe it won't as much as I think.�

This version will also include the new CAPTCHA stuff and the ability to tune mailings for specific domains - so for example, if you don't want to use, say, the VERP feature for Yahoo mail folks (but you do for everyone else), you don't have to. Kind of cool. Finally, this alpha will include support for social bookmarking/link sharing button/badges on the bottom of the archived message screens, like this:�



Currently, there's support for http://del.icio.us� http://digg.com http://newsvine.com http://reddit.com Yahoo, http://ma.gnolia.com and about 5 lesser known ones - this'll make it easy for people to share your archived messages with other peeps. Here's an example of a archived message shared on del.icio.us:�

So, if you search on del.icio.us for "CAPTCHA and dada" you'll find that entry:�


Kind of neat.�

Also, all those little image badges, as well as the feed badge will be generated by Dada Mail, so there's no extra step to put the images somewhere. Also, kind of neat.�



Also on Monday, I'll be releasing version 2.10.10 of the program, which will be nothing but bug fixes, over -� WAY over 50 of them, so it's a decent upgrade - also has some security fixes. the 2.11 alpha will contain most of these bug fixes as well.�



Any questions? :)




--�
Justin Simoni�

.: Dada Mail�"Write Once - Distribute Everywhere" Email Communication Software�� � �� �

aolim: leaddadaist

Post:
mailto:dadadev@PROTECTED

Unsubscribe:
http://mojo.skazat.com/cgi-bin/dada/mail.cgi/u/[list]/

List Information:
http://mojo.skazat.com/cgi-bin/dada/mail.cgi/list/[list]

Archive:
http://mojo.skazat.com/cgi-bin/dada/mail.cgi/archive/[list]

rss/atom
‹ (no subject)
Index
RE: Auto-Pickup of Mailings (Big update on next major version 2.11) ›
  • This mailing list is a public mailing list - anyone may join or leave, at any time.
  • This mailing list is a group discussion list (unmoderated)

  • Start a new thread, email: dadadev@dadamailproject.com

This is the developer discussion mailing list for Dada Mail.

If you are just looking for support Dada Mail, consult the message boards at:

https://forum.dadamailproject.com

Documentation for Dada Mail:

https://dadamailproject.com/d

Specifically, see the Error FAQ:

https://dadamailproject.com/d/FAQ-errors.pod.html

To post to this list, send a message to:

mailto:dadadev@dadamailproject.com

All subscribers of this list may post to the list itself.

Topics that are welcome:

  • Constructive critiques on the program (I like, "x", but, "y" needs some work - here's an idea on how to make this better...)
  • Bug/Error reports
  • Bug fixes
  • Request For Comments on any changes to the program
  • Help customizing Dada Mail for your own needs
  • Patches
  • Language Translations
  • Support Documentation/Doc editing, FAQ's, etc.
  • Discussion of any changes that you would like to be committed to the next version of Dada Mail -

Dada Mail is on Github:

https://github.com/justingit/dada-mail/

If you would like to fork, branch, send over PRs, open up issues, etc.

Privacy Policy:

This Privacy Policy is for this mailing list, and this mailing list only.

Email addresses collection through this mailing list are used explicitly to work within this email discussion list.

We only collect email addresses through our Closed-Loop Opt-In system.

We don't use your email address for any other purpose.

We won't be sharing your email address with any other entity.

Unsubscription can be done at any time. Please contact us at: justin@dadamailproject.com for any help regarding your subscription, including removal from the mailing list.

All mailing list messages sent from us will include a subscription removal link, which will allow you to remove yourself from this mailing list automatically, and permanently.

All consent to use your email address for any other purpose stated at the time of the mailing list subscription will also be revoked upon mailing list removal.