Re: Slowdowns again

 
From: "John Collins" <john@PROTECTED>
Date: February 15th 2010
Justin J wrote:
There's two features in Dada Mail that can make this faster. One is the,
	Adjust Sleep Time

feature, that tries to remove some of the time actual message sending takes from the batch time, so that the sending time for the entire mailing list is more similar to the estimate. Since you have such a huge batch, and such a small batch time, you're not going to see much difference. 

The other is 

	Use One SMTP Connection per Batch (Experimental)

Which allows you to send a batch with just one connection to the mail server. Probably supposed to be faster, since you skip the connect/disconnect parts. I wouldn't do it with a batch of 400. Maybe more like 30. 

  

Tried those settings on the first part of the trial.  I found that SMTP mailing with those exact settings  took about 50% longer.  Here are the stats.

On 2/13
(SMTP sending.  batch of 400, 10sec delay between batches, single SMTP connection per batch)
Total Mailing time, 11hrs, 7 minutes
43 messages per minute


On 2/7
Total Mailing Time: 7 hours, 23 minutes, 37 seconds
(SENDMAIL, no batches )
65 messages per minute


Justin J wrote:
You can see how fast messages can send out in Dada Mail by changing the $MASS_MAIL_SETTINGS from their default: 
	$MASS_MAIL_SETTINGS = "|$MAILPROG -t";
to, 
	$MASS_MAIL_SETTINGS = ">>$LOGS/test_mail.txt";

(Put those two lines in your .dada_config file)

That'll just write the email messages you want to send to a file, instead of to the mail server. Set your list to use sendmail, test to make sure the emails aren't being sent out, but rather being written to this file and when you're all ready, send out a message, as normal. You're *regular* email messages - subscriptions/unsubscriptions and the finishing email notification will still be sent - so send a test message to the list owner - that should be written to the file.  

Whatever speed Dada Mail can send things out, is the top speed you'll probably ever be able to get. 

Subtract that speed from another sending, hopefully, using the same list, message, etc and you have the time it takes for Dada Mail to talk to the mail server. 


My guess is that Dada Mail going through the addresses and writing out the emails will be pretty fast, but not *blazing*  - how fast is it to just template out an admin screen (it's the same templating engine!), and the mail server will be a little slower. Just a guess, though. 

  

That's a good and revealing test.  Actually it didn't turn out the way expected. 

The result is that the email server is processing and sending the messages about as fast as they are fed to it.

Writing to file produced these two stats using two different sized emails
180kb message came out about the same as the real message I sent yesterday
2,451 per hour or 40.85 per minute

A small test message of 10kb wrote the text files at the rate of
22,967 per hour or 382.78 per minute

Not a surprise that it takes longer to send a larger message.  Compiling time appears to be almost a linear increase from the 10kb msg to the 180kb msg.
(As a further test, I then tripled the size of the mssage to 540kb  and found that it dropped to 377msgs per hour or about one every six seconds)

The new way of handling click-throughs in a mysql table in 4.03 precludes me from running the same test again using Dada 3.04 today.  The click-through links on the v4.x msg sent yesterday probably wouldn't work on v3.x and I can't have that happen on a Monday when most readers are viewing the newsletter, but I can try it at the end of the week.


-------------------------------------------------
John Collins
Meetings and Mixers
Box 80461
Rancho Santa Margarita, CA 92688
c949 689 7070
john@PROTECTED
http://www.meetingsandmixers.com/

Post:
mailto:[list_settings.discussion_pop_email]

Unsubscribe:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev/

List Information:
[PROGRAM_URL]/list/[list_settings.list]

Archive:
[PROGRAM_URL]/archive/[list_settings.list]

Developer Info:
http://dev.dadamailproject.com

Mailing List Powered by Dada Mail

  • This mailing list is a public mailing list - anyone may join or leave, at any time.
  • This mailing list is a group discussion list (unmoderated)
  • Start a new thread, email: dadadev@dadamailproject.com

This is the developer discussion mailing list for Dada Mail.

If you are just looking for support Dada Mail, consult the message boards at:

https://forum.dadamailproject.com

Documentation for Dada Mail:

https://dadamailproject.com/d

Specifically, see the Error FAQ:

https://dadamailproject.com/d/FAQ-errors.pod.html

To post to this list, send a message to:

mailto:dadadev@dadamailproject.com

All subscribers of this list may post to the list itself.

Topics that are welcome:

  • Constructive critiques on the program (I like, "x", but, "y" needs some work - here's an idea on how to make this better...)
  • Bug/Error reports
  • Bug fixes
  • Request For Comments on any changes to the program
  • Help customizing Dada Mail for your own needs
  • Patches
  • Language Translations
  • Support Documentation/Doc editing, FAQ's, etc.
  • Discussion of any changes that you would like to be committed to the next version of Dada Mail -

Dada Mail is on Github:

https://github.com/justingit/dada-mail/

If you would like to fork, branch, send over PRs, open up issues, etc.

Privacy Policy:

This Privacy Policy is for this mailing list, and this mailing list only.

Email addresses collection through this mailing list are used explicitly to work within this email discussion list.

We only collect email addresses through our Closed-Loop Opt-In system.

We don't use your email address for any other purpose.

We won't be sharing your email address with any other entity.

Unsubscription can be done at any time. Please contact us at: justin@dadamailproject.com for any help regarding your subscription, including removal from the mailing list.

All mailing list messages sent from us will include a subscription removal link, which will allow you to remove yourself from this mailing list automatically, and permanently.

All consent to use your email address for any other purpose stated at the time of the mailing list subscription will also be revoked upon mailing list removal.