Real Software Forums

The forum for Real Studio and other Real Software products.
[ REAL Software Website | Board Index ]
It is currently Sat Aug 17, 2019 6:36 am
xojo

All times are UTC - 5 hours




Post new topic Reply to topic  [ 11 posts ] 
Author Message
 Post subject: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Tue Aug 28, 2012 5:09 pm 
Offline
User avatar

Joined: Sat Jan 29, 2011 6:24 pm
Posts: 33
Location: Germany, Europe
Hello,

the following question was already posted a few days ago in the mailing list... but I guess that the announcement of new RB version got more attention at this time. :-)

I have a small application that is going to use a SMTPSocket for sending mails to a mailing list. The mailing list currently contains about 500 subscribers, but a second one with several thousand subscribers is planned. Each mailing may consist of about 50 KB of text/HTML and a 2 MB attachment (PDF catalogue).

Since each mail is customized (by taking some data out of a SQL database), I can't use BCC addressing scheme, so I really have 500 (or more) unique EMailMessage objects to be sent.

How many messages can I safely add to the message queue of a SMTPSocket object at once before calling the SendMail method ?

Or should I better sent one message at a time ? But wouldn't this mean that for each mail, the SMTP server authentication must be handled, which may take some time slowing down the entire process ?

Has someone already made experiences with such heavy use of SMTPSocket in RB ? Anything to take care about or consider ? Advice, comments, tips - whatever you have for me, it's highly appreciated ! :-)

Tobias.


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Tue Aug 28, 2012 9:21 pm 
Offline
User avatar

Joined: Fri Jan 12, 2007 10:59 am
Posts: 136
Location: Victoriaville, QC, Canada
I haven't sent that many emails, but before I attempt an answer, I'd like to know a few things: do you need to complete the entire process in a certain amount of time? How often will you be sending your thousands of emails? Once a day? a week? a month?

_________________
Bruno Fréchette
PJJ Productions inc.
RealStudio Web & Desktop on OSX


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 4:59 am 
Offline
User avatar

Joined: Sat Jan 29, 2011 6:24 pm
Posts: 33
Location: Germany, Europe
Time is not an important factor here, because it's a desktop app that may run minimized in the background. Reliability goes first. So I thought to send one mail at a time, even this means to re-authenticate after each message (at least, if I understood the way SMTPSocket works correctly). Nevertheless, the task must be completed in a reasonable amount of time.

How often are mailings sent ? Not sure, currently it's planned weekly. But that should not matter finding the right solution, should ?

But that brings me to another question: How is SMTP configured by common service providers ? Are there any issues to take care about ? Perhaps somebody is experienced in SMTP server maintenance and can say a word about this topic as well. Because the solution should work with a "normally configured" business mail account.

Oh, by the way to avoid misunderstandings: The project I'm working on is for a small business. It's all serious, no UCE. :-)


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 8:33 am 
Offline
User avatar

Joined: Fri Jan 12, 2007 10:59 am
Posts: 136
Location: Victoriaville, QC, Canada
You're right about finding the right solution. I apologize for all the questions. What I would do since time is not of the essence (up to a certain point) is start with a one-at-a-time solution. You'll see how long it takes and if optimization is necessary.

But I don't have the SMTP knowledge you're looking for.

Anyone else?

_________________
Bruno Fréchette
PJJ Productions inc.
RealStudio Web & Desktop on OSX


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 12:44 pm 
Offline

Joined: Fri Jan 06, 2006 3:21 pm
Posts: 12388
Location: Portland, OR USA
Talk to your service provider. They typically will shut you down if you send too much mail too quickly, because it looks like spam. You may be able to pay more and get an exception. Or use a bulk mail service instead.


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 12:52 pm 
Offline
User avatar

Joined: Sat Jan 29, 2011 6:24 pm
Posts: 33
Location: Germany, Europe
Thanks for your hint; but it does not answer my original question. :-)


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 1:04 pm 
Offline
User avatar

Joined: Fri Jan 12, 2007 10:59 am
Posts: 136
Location: Victoriaville, QC, Canada
Come to think of it, you could setup your own mail server and see how it behaves with sending multiple mails. You could create a couple of users and generate a list of a few thousand emails and try to send them all at the same time to see what happens. I have my own email serves, I think that's what I would do.

_________________
Bruno Fréchette
PJJ Productions inc.
RealStudio Web & Desktop on OSX


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 3:33 pm 
Offline
User avatar

Joined: Sat Jan 29, 2011 6:24 pm
Posts: 33
Location: Germany, Europe
Yes, this should be no problem. I have access to two virtual servers, which I could modify and give it a try. However, the mail server would deliver most messages locally. So even if I grab some outside e-mail addresses, it will never reach the real world scenario.

And because I don't know how my client is using the software in the future, I look for the most compatible method. So while one solution works well on a modified mail server, it could cause trouble on another one.

So, I hoped that someone of you already had to deal with a similar project in the past and can share some experiences.

Asked directly: If you are in need to send a huge set of e-mails, how many would you put into the message queue before sending it ? Could be ten a good number to start with, then force the software to wait ten seconds and then send another set of ten mails ? I could put these numbers into the configuration section of the software, so the user could adjust them according their (SMTP service) needs.


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 3:57 pm 
Offline
User avatar

Joined: Fri Jan 12, 2007 10:59 am
Posts: 136
Location: Victoriaville, QC, Canada
A quick search turned the following document:http://www.websightdesign.com/views/media/WSD_MailingCampaignRules.pdf

one paragraph says:

Quote:
Every major service provider has their own threshold of how many emails can be sent at one time from their servers before you are considered sending spam, and another threshold for how many mailings can come from one source into their systems at a time before the source is considered as sending spam.
Unfortunately these numbers change and when they do, service providers do not easily publicize this fact. Some providers will block you if they get 10 emails in one mailing, others block you if they get 200, 250, or even 500. Every provider is different and they routinely change their threshold.
Based on our experience, we recommend that you send out no more than 10 emails at any one time, at five or even ten minute intervals, and no more than 100 in any hour. Sending at any higher rate is strongly discouraged.


So i guess your figures are pretty correct: 10 at a time, 10 minutes interval. So this gives you a rate of about 60/hour, or 1440/day.

But...

this is only if your software actually relies on your service provider to relay the emails to the outside world. If you setup your own smtp server, with full domain name and correct reverse, your server will then talk directly to each recipient's email server and not rely on a single point of exit (your provider's outgoing email server). Of course, this requires a more elaborate internet setup with fixed ip & dns. And because your email server can be considered a "spammer" and become blacklisted, it still doesn't mean you can flood the net. But it will give you a lot more flexibility.

So in the end, I believe your question is more about bulk email best practices than about technical capabilities. I think the best practices will be waaaay below any technical limitation you may encounter. :wink:

_________________
Bruno Fréchette
PJJ Productions inc.
RealStudio Web & Desktop on OSX


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Thu Aug 30, 2012 4:38 pm 
Offline
User avatar

Joined: Sat Sep 30, 2006 10:07 pm
Posts: 32
Location: Hertfordshire, England
Something to bear in mind: 3000 recipients × 2 MB x 1.33 (for base64 encoding) comes out as 8 GB of mail traffic.

I don't know your specifics, but since you're running server software on the desktop in a programming framework that supports Windows services, I presume that there is no IT deparment available here to set up bandwidth management on the router and deal with a local node bringing the WAN link to a complete standstill; this is what I see every time anyone tries doing what you're doing. (If you do have the traffic capacity, I've found Arclab MailList Controller to be pretty decent, and you can run the mailer service on your own PC. This is not a wheel that needs reinventing.)

I would honestly suggest that you're better off signing up with a professional mailing service instead, whose job it would be to deal with IP reputation issues, SPF records, correct PTR configuration, bounce management, Yahoo! feedback loops, DomainKeys/DKIM, and all the other hassle that goes with sending mailouts. (And it keeps this distinct from your own mail setup.)


Top
 Profile  
Reply with quote  
 Post subject: Re: Heavy use of SMTPSocket... how to avoid problems ?
PostPosted: Tue Sep 04, 2012 9:00 am 
Offline
User avatar

Joined: Sat Nov 11, 2006 2:43 pm
Posts: 1221
Location: This poster has left the forums
If you don't want to use a off-the-shelf mailing list provider, you could run a mail server locally (sendmail or Postfix for example).

As for the large attachments, it may make sense to host the attachment externally and provide a link in the email. Given the vast array of devices receiving email, your clients may not be grateful that your email attachment hits every device, especially if they pay for bandwidth.

_________________
%Invalidforumsignatureexception% user signature not found


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 5 hours


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group