Re: Template Optimizations/Enhancements
Before
testing it, I would have to be sure that I would not be changing
something that would prevent me from going back to the earlier version.
(if it ain't broke, etc)
The improvement that has me the most interested in upgrading has not been implemented yet. It's the changes suggested to Bounce handler in which soft bounces are cleared if mail to a recipient is accepted on subsequent mailings. There are other nice features, but I've gotten by without them so far.
But I'd even forego that feature if I had to give up a lot of speed to get it. Your sample of an 180kb message is about my norm for my weekly mailer. It's mostly html formatted text with a few small logos that reside on my server. In addition to the large one, I have sometimes sent two more smaller ones on the same day. So I need the speed, they should be received by the subscribers at close to the same time of day. I can send the 25,000 in about an hour with the old version so I can get all three out in 3 - 4 hours. Close enough.
By your tests as indicated here
http://dadamailproject.com/images/dev/html_template_pro_optimizations.jpg
You state that you are saving 8 secs for 10 messages, extrapolating to a savings of 5.5hrs for a 25000 list.
Maybe I'm missing something, but looking at the other column, it appears to indicate that it took 16secs to send ten. Therefore, 16secs * 2500 / 3600 secs/hr = 11.1 hrs. Ooops.
Whereas I can send ten in 1.05 secs based on the 7 per second rate I am getting.
I use a Mac so I *think* I can ssh to the server using Terminal, but I've never done it before so I'd need some direction.
I'm intrigued, but I need to know more.
John
-------------------------------------------------
John Collins
In The Calendar.com
c949 689 7070
john@PROTECTED
http://www.inthecalendar.com
The improvement that has me the most interested in upgrading has not been implemented yet. It's the changes suggested to Bounce handler in which soft bounces are cleared if mail to a recipient is accepted on subsequent mailings. There are other nice features, but I've gotten by without them so far.
But I'd even forego that feature if I had to give up a lot of speed to get it. Your sample of an 180kb message is about my norm for my weekly mailer. It's mostly html formatted text with a few small logos that reside on my server. In addition to the large one, I have sometimes sent two more smaller ones on the same day. So I need the speed, they should be received by the subscribers at close to the same time of day. I can send the 25,000 in about an hour with the old version so I can get all three out in 3 - 4 hours. Close enough.
By your tests as indicated here
http://dadamailproject.com/images/dev/html_template_pro_optimizations.jpg
You state that you are saving 8 secs for 10 messages, extrapolating to a savings of 5.5hrs for a 25000 list.
Maybe I'm missing something, but looking at the other column, it appears to indicate that it took 16secs to send ten. Therefore, 16secs * 2500 / 3600 secs/hr = 11.1 hrs. Ooops.
Whereas I can send ten in 1.05 secs based on the 7 per second rate I am getting.
I use a Mac so I *think* I can ssh to the server using Terminal, but I've never done it before so I'd need some direction.
I'm intrigued, but I need to know more.
John
Justin J
November 9, 2011 8:40 PMOn Nov 9, 2011, at 6:31 PM, Justin J wrote:But take in account how fast the templating engine is now, and how fast my tests say it could be. The savings just creating the email message would be a few minutes to... I dunno, an hour? per *25,000*. I don't know if the template engine is really a large bottle neck.Actually, depending on what I use to look into this, I get different answers. Using a tool called Devel::NYTProf (which is pretty incredible) it actually gives a dramatic speed improvement: http://dadamailproject.com/images/dev/html_template_pro_optimizations.jpg The top table is using HTML::Template, the bottom table is using HTML::Template::Pro and I tested just sending out a message to 10 people. The message is around 180 KB (so a very large message). It's showing a speedup of over 8 seconds. In that case, speeding up just that little bit for 25k subscribers would yield around 5 1/2 hours faster to send, if my math is correct. That's not bad, for such an easy optimization. Obv., YMMV depending a lot on the type of message you're sending.Justin J November 9, 2011 4:25 PM Hello Everyone, I've been toying (again?) with changing the template engine in Dada Mail (but keeping the syntax the same) from HTML::Template: http://search.cpan.org/dist/HTML-Template/ To HTML::Template::Pro http://search.cpan.org/dist/HTML-Template-Pro/ HTML::Template::Pro promises speedups between 10 and 25x over HTML::Template, which is pretty nice. The downside is that I won't be able to ship Dada Mail with HTML::Template::Pro enabled by default, as HTML::Template;:Pro requires you to compile the module. I can still make it an option for those looking for a speed/memory use improvement. Doing a quick test, I'm able to see around a 10x speed improvement over what is currently being used when making HTML screens in Dada Mail, and a 2.7x speedup for parsing templates in email messages. That seems a fairly good improvement, but the initial speeds of HTML::Template aren't super slow - for HTML screens, we're talking about .025 seconds to render the screen with HTML::Template, and .0025 seconds to render the screen with HTML::Template::Pro. I'm not sure of memory usage, but I'm guessing that memory using is lower with HTML::Template::Pro. You can follow this part of the project here: https://github.com/justingit/dada-mail/tree/features-html_template_pro And actually, all the different sub projects I have happening here: https://github.com/justingit/dada-mail/branches Looks as if v4.9.0 is going to be a big release.------------------------------------------------- John Collins In The Calendar.com c949 689 7070 john@PROTECTED http://www.inthecalendar.com Post: mailto:dadadev@PROTECTED Unsubscribe: http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev/ List Information: http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadev Archive: http://dadamailproject.com/cgi-bin/dada/mail.cgi/archive/dadadev Developer Info: http://dev.dadamailproject.com Mailing List Powered by Dada Mail-- * Post: mailto:dadadev@PROTECTED * Unsubscribe: http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev * List Information: http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadev/ * Archive: http://dadamailproject.com/cgi-bin/dada/mail.cgi/archive/dadadev Developer Info: http://dev.dadamailproject.com Mailing List Powered by Dada Mail http://dadamailproject.com/cgi-bin/dada/mail.cgi/what_is_dada_mail/
Justin J
November 9, 2011 5:31 PMI reverted to 2.11.16 quite some time ago because it could send out 25,000 emails about 180kb in size in about an hour, which is about 7 per second. I needed the speed because sometimes I send several in a day. Subsequent versions took much longer to send. I'd like to have the advanced features of the newer versions and would update in a heartbeat if the speeds were comparable.But take in account how fast the templating engine is now, and how fast my tests say it could be. The savings just creating the email message would be a few minutes to... I dunno, an hour? per *25,000*. I don't know if the template engine is really a large bottle neck. I sincerely think your mad :) to be using any version of 2.x of Dada Mail. Your missing out on way too many things that have been created that would work in your favor since that ancient release. I'd strongly encourage you to try out the version of Dada Mail with this feature enabled and I'd really really encourage you to sponsor the feature, to get it out the door faster :) Remember, we'll need access to the command line (via ssh) and a compiler to get HTML::Template::Pro installed.Justin J November 9, 2011 4:25 PM Hello Everyone, I've been toying (again?) with changing the template engine in Dada Mail (but keeping the syntax the same) from HTML::Template: http://search.cpan.org/dist/HTML-Template/ To HTML::Template::Pro http://search.cpan.org/dist/HTML-Template-Pro/ HTML::Template::Pro promises speedups between 10 and 25x over HTML::Template, which is pretty nice. The downside is that I won't be able to ship Dada Mail with HTML::Template::Pro enabled by default, as HTML::Template;:Pro requires you to compile the module. I can still make it an option for those looking for a speed/memory use improvement. Doing a quick test, I'm able to see around a 10x speed improvement over what is currently being used when making HTML screens in Dada Mail, and a 2.7x speedup for parsing templates in email messages. That seems a fairly good improvement, but the initial speeds of HTML::Template aren't super slow - for HTML screens, we're talking about .025 seconds to render the screen with HTML::Template, and .0025 seconds to render the screen with HTML::Template::Pro. I'm not sure of memory usage, but I'm guessing that memory using is lower with HTML::Template::Pro. You can follow this part of the project here: https://github.com/justingit/dada-mail/tree/features-html_template_pro And actually, all the different sub projects I have happening here: https://github.com/justingit/dada-mail/branches Looks as if v4.9.0 is going to be a big release.------------------------------------------------- John Collins In The Calendar.com c949 689 7070 john@PROTECTED http://www.inthecalendar.com Post: mailto:dadadev@PROTECTED Unsubscribe: http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev/ List Information: http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadev Archive: http://dadamailproject.com/cgi-bin/dada/mail.cgi/archive/dadadev Developer Info: http://dev.dadamailproject.com Mailing List Powered by Dada Mail
MM John
November 9, 2011 5:06 PM
Re: Template Optimizations/Enhancements You got my attention on this one.
I reverted to 2.11.16 quite some time ago because it could send out 25,000 emails about 180kb in size in about an hour, which is about 7 per second. I needed the speed because sometimes I send several in a day. Subsequent versions took much longer to send. I'd like to have the advanced features of the newer versions and would update in a heartbeat if the speeds were comparable.
FYI: using sendmail, no batches, no gaps; just one big gulp. It's faster than smtp sending for this particular version.
John CollinsPost:
mailto:dadadev@PROTECTEDUnsubscribe:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/u/dadadev/List Information:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/list/dadadevArchive:
http://dadamailproject.com/cgi-bin/dada/mail.cgi/archive/dadadevDeveloper Info:
http://dev.dadamailproject.comMailing List Powered by Dada Mail
Mailing List Powered by Dada Mail
Justin J
November 9, 2011 4:25 PMHello Everyone,
I've been toying (again?) with changing the template engine in Dada Mail (but keeping the syntax the same) from HTML::Template:
http://search.cpan.org/dist/HTML-Template/
To HTML::Template::Pro
http://search.cpan.org/dist/HTML-Template-Pro/
HTML::Template::Pro promises speedups between 10 and 25x over HTML::Template, which is pretty nice. The downside is that I won't be able to ship Dada Mail with HTML::Template::Pro enabled by default, as HTML::Template;:Pro requires you to compile the module. I can still make it an option for those looking for a speed/memory use improvement.
Doing a quick test, I'm able to see around a 10x speed improvement over what is currently being used when making HTML screens in Dada Mail, and a 2.7x speedup for parsing templates in email messages.
That seems a fairly good improvement, but the initial speeds of HTML::Template aren't super slow - for HTML screens, we're talking about .025 seconds to render the screen with HTML::Template, and .0025 seconds to render the screen with HTML::Template::Pro. I'm not sure of memory usage, but I'm guessing that memory using is lower with HTML::Template::Pro.
You can follow this part of the project here:
https://github.com/justingit/dada-mail/tree/features-html_template_pro
And actually, all the different sub projects I have happening here:
https://github.com/justingit/dada-mail/branches
Looks as if v4.9.0 is going to be a big release.
-------------------------------------------------
John Collins
In The Calendar.com
c949 689 7070
john@PROTECTED
http://www.inthecalendar.com
-------------------------------------------------
John Collins
In The Calendar.com
c949 689 7070
john@PROTECTED
http://www.inthecalendar.com
- This mailing list is a public mailing list - anyone may join or leave, at any time.
- This mailing list is a group discussion list (unmoderated)
-
Start a new thread, email: dadadev@dadamailproject.com
This is the developer discussion mailing list for Dada Mail.
If you are just looking for support Dada Mail, consult the message boards at:
https://forum.dadamailproject.com
Documentation for Dada Mail:
Specifically, see the Error FAQ:
https://dadamailproject.com/d/FAQ-errors.pod.html
To post to this list, send a message to:
mailto:dadadev@dadamailproject.com
All subscribers of this list may post to the list itself.
Topics that are welcome:
- Constructive critiques on the program (I like, "x", but, "y" needs some work - here's an idea on how to make this better...)
- Bug/Error reports
- Bug fixes
- Request For Comments on any changes to the program
- Help customizing Dada Mail for your own needs
- Patches
- Language Translations
- Support Documentation/Doc editing, FAQ's, etc.
- Discussion of any changes that you would like to be committed to the next version of Dada Mail -
Dada Mail is on Github:
https://github.com/justingit/dada-mail/
If you would like to fork, branch, send over PRs, open up issues, etc.
Privacy Policy:
This Privacy Policy is for this mailing list, and this mailing list only.
Email addresses collection through this mailing list are used explicitly to work within this email discussion list.
We only collect email addresses through our Closed-Loop Opt-In system.
We don't use your email address for any other purpose.
We won't be sharing your email address with any other entity.
Unsubscription can be done at any time. Please contact us at: justin@dadamailproject.com for any help regarding your subscription, including removal from the mailing list.
All mailing list messages sent from us will include a subscription removal link, which will allow you to remove yourself from this mailing list automatically, and permanently.
All consent to use your email address for any other purpose stated at the time of the mailing list subscription will also be revoked upon mailing list removal.