Real Software Forums
http://forums.realsoftware.com/

Heavy use of SMTPSocket... how to avoid problems ?
http://forums.realsoftware.com/viewtopic.php?f=2&t=45156
Page 1 of 1

Author:  tobiaseichner [ Tue Aug 28, 2012 5:09 pm ]
Post subject:  Heavy use of SMTPSocket... how to avoid problems ?

Hello,

the following question was already posted a few days ago in the mailing list... but I guess that the announcement of new RB version got more attention at this time. :-)

I have a small application that is going to use a SMTPSocket for sending mails to a mailing list. The mailing list currently contains about 500 subscribers, but a second one with several thousand subscribers is planned. Each mailing may consist of about 50 KB of text/HTML and a 2 MB attachment (PDF catalogue).

Since each mail is customized (by taking some data out of a SQL database), I can't use BCC addressing scheme, so I really have 500 (or more) unique EMailMessage objects to be sent.

How many messages can I safely add to the message queue of a SMTPSocket object at once before calling the SendMail method ?

Or should I better sent one message at a time ? But wouldn't this mean that for each mail, the SMTP server authentication must be handled, which may take some time slowing down the entire process ?

Has someone already made experiences with such heavy use of SMTPSocket in RB ? Anything to take care about or consider ? Advice, comments, tips - whatever you have for me, it's highly appreciated ! :-)

Tobias.

Author:  BrunoFrechette [ Tue Aug 28, 2012 9:21 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

I haven't sent that many emails, but before I attempt an answer, I'd like to know a few things: do you need to complete the entire process in a certain amount of time? How often will you be sending your thousands of emails? Once a day? a week? a month?

Author:  tobiaseichner [ Thu Aug 30, 2012 4:59 am ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Time is not an important factor here, because it's a desktop app that may run minimized in the background. Reliability goes first. So I thought to send one mail at a time, even this means to re-authenticate after each message (at least, if I understood the way SMTPSocket works correctly). Nevertheless, the task must be completed in a reasonable amount of time.

How often are mailings sent ? Not sure, currently it's planned weekly. But that should not matter finding the right solution, should ?

But that brings me to another question: How is SMTP configured by common service providers ? Are there any issues to take care about ? Perhaps somebody is experienced in SMTP server maintenance and can say a word about this topic as well. Because the solution should work with a "normally configured" business mail account.

Oh, by the way to avoid misunderstandings: The project I'm working on is for a small business. It's all serious, no UCE. :-)

Author:  BrunoFrechette [ Thu Aug 30, 2012 8:33 am ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

You're right about finding the right solution. I apologize for all the questions. What I would do since time is not of the essence (up to a certain point) is start with a one-at-a-time solution. You'll see how long it takes and if optimization is necessary.

But I don't have the SMTP knowledge you're looking for.

Anyone else?

Author:  timhare [ Thu Aug 30, 2012 12:44 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Talk to your service provider. They typically will shut you down if you send too much mail too quickly, because it looks like spam. You may be able to pay more and get an exception. Or use a bulk mail service instead.

Author:  tobiaseichner [ Thu Aug 30, 2012 12:52 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Thanks for your hint; but it does not answer my original question. :-)

Author:  BrunoFrechette [ Thu Aug 30, 2012 1:04 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Come to think of it, you could setup your own mail server and see how it behaves with sending multiple mails. You could create a couple of users and generate a list of a few thousand emails and try to send them all at the same time to see what happens. I have my own email serves, I think that's what I would do.

Author:  tobiaseichner [ Thu Aug 30, 2012 3:33 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Yes, this should be no problem. I have access to two virtual servers, which I could modify and give it a try. However, the mail server would deliver most messages locally. So even if I grab some outside e-mail addresses, it will never reach the real world scenario.

And because I don't know how my client is using the software in the future, I look for the most compatible method. So while one solution works well on a modified mail server, it could cause trouble on another one.

So, I hoped that someone of you already had to deal with a similar project in the past and can share some experiences.

Asked directly: If you are in need to send a huge set of e-mails, how many would you put into the message queue before sending it ? Could be ten a good number to start with, then force the software to wait ten seconds and then send another set of ten mails ? I could put these numbers into the configuration section of the software, so the user could adjust them according their (SMTP service) needs.

Author:  BrunoFrechette [ Thu Aug 30, 2012 3:57 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

A quick search turned the following document:http://www.websightdesign.com/views/media/WSD_MailingCampaignRules.pdf

one paragraph says:

Quote:
Every major service provider has their own threshold of how many emails can be sent at one time from their servers before you are considered sending spam, and another threshold for how many mailings can come from one source into their systems at a time before the source is considered as sending spam.
Unfortunately these numbers change and when they do, service providers do not easily publicize this fact. Some providers will block you if they get 10 emails in one mailing, others block you if they get 200, 250, or even 500. Every provider is different and they routinely change their threshold.
Based on our experience, we recommend that you send out no more than 10 emails at any one time, at five or even ten minute intervals, and no more than 100 in any hour. Sending at any higher rate is strongly discouraged.


So i guess your figures are pretty correct: 10 at a time, 10 minutes interval. So this gives you a rate of about 60/hour, or 1440/day.

But...

this is only if your software actually relies on your service provider to relay the emails to the outside world. If you setup your own smtp server, with full domain name and correct reverse, your server will then talk directly to each recipient's email server and not rely on a single point of exit (your provider's outgoing email server). Of course, this requires a more elaborate internet setup with fixed ip & dns. And because your email server can be considered a "spammer" and become blacklisted, it still doesn't mean you can flood the net. But it will give you a lot more flexibility.

So in the end, I believe your question is more about bulk email best practices than about technical capabilities. I think the best practices will be waaaay below any technical limitation you may encounter. :wink:

Author:  Telcontar [ Thu Aug 30, 2012 4:38 pm ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

Something to bear in mind: 3000 recipients × 2 MB x 1.33 (for base64 encoding) comes out as 8 GB of mail traffic.

I don't know your specifics, but since you're running server software on the desktop in a programming framework that supports Windows services, I presume that there is no IT deparment available here to set up bandwidth management on the router and deal with a local node bringing the WAN link to a complete standstill; this is what I see every time anyone tries doing what you're doing. (If you do have the traffic capacity, I've found Arclab MailList Controller to be pretty decent, and you can run the mailer service on your own PC. This is not a wheel that needs reinventing.)

I would honestly suggest that you're better off signing up with a professional mailing service instead, whose job it would be to deal with IP reputation issues, SPF records, correct PTR configuration, bounce management, Yahoo! feedback loops, DomainKeys/DKIM, and all the other hassle that goes with sending mailouts. (And it keeps this distinct from your own mail setup.)

Author:  pony [ Tue Sep 04, 2012 9:00 am ]
Post subject:  Re: Heavy use of SMTPSocket... how to avoid problems ?

If you don't want to use a off-the-shelf mailing list provider, you could run a mail server locally (sendmail or Postfix for example).

As for the large attachments, it may make sense to host the attachment externally and provide a link in the email. Given the vast array of devices receiving email, your clients may not be grateful that your email attachment hits every device, especially if they pay for bandwidth.

Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/