Sendy sended 50 times to same contact

Comments

• Ben

  December 2012 edited December 2012

  Hi Thomás,
  
  Sorry to hear that.
  
  Thanks for the screenshots. I saw that your 'sends left' is in a negative number. These numbers are retrieved from your Amazon SES account via their API. Having said that, it is unlikely to reach a 'negative' number since Amazon SES will throw an 'ThrottlingException' if you send over your limit as this new blog post on Amazon's website tells us http://sesblog.amazon.com/post/Tx8YGT0YZ9SQLD/What-Happens-When-You-Reach-Your-Sending-Limits
  
  I've tested the sending code rigorously, Sendy will not send 50 emails to the same contact. For each email, it will send once, if Amazon returns a failed error code, Sendy will retry in 1 second. If it fails again, it will not bother to send anymore. The erroneous emails will then be consolidated as a CSV download at the end of the campaign.
  
  Although I doubt this, do you have multiple cron jobs set up to execute the scheduled.php script? Are you using cron to send and is it set to execute at */5?
  
  Would you mind sending me your installation and FTP credentials to do a check? I understand you trust Amazon more because they are huge. But they did had issues previously sending emails to the same user thousands of times https://forums.aws.amazon.com/thread.jspa?messageID=400376&#400376
  
  I do care about my users as much as Amazon care about theirs, just that I have not receive reports of emails being sent to the same contact over 50 times. I don't have the same problem as well when sending to thousands and thousands of subscribers in order to fix a problem. I understand from your Forrst post that you are going to build your own email marketing system with Rails and probably don't care about Sendy anymore. But if you could let me find out why it's working like this on your installation, that'd explain.
  
  Thanks.
• thomashs

  December 2012

  Crontab is set to */5.
  Anyway, you should not count on cron jobs / timeouts to change logic on the application.
  Consider that if timeout / shutdown occurs, you would send twice to the queue table.
  By the way, the other day my amazon instance was running of disk space because the queue table kept growing to millions of records.
• thomashs

  December 2012

  Sorry, but I can't give you FTP credentials right now.
  Can't you release a version of Sendy with log records on critical functions?
  It would be easier to investigate problems..
• thomashs

  December 2012

  Oh! And if you take a look at the print screen I sent you, the recipients count is far more than the list size which is 14.565 .. And yes, I'm still interested on Sendy - Developing an e-mail marketing platform will take weeks, and we will start the project only in frebuary - Although we are said about this particular incident, I believe we can work things out and learn from each other. Thanks in advance.
• shadow

  December 2012

  Hi Ben, we had this same problem the other day as mentioned on this thread.
  http://sendy.co/discussion/comment/1309#Comment_1309
  
  We had a negative SES quota shown on the Amazon SES console after we sent out the campaign.
  
  I suspect that it was queued more than once for one of the newsletters. It was lucky that it was only queued twice rather than times 50 as above hence we did not get any complains for that.
• Ben

  December 2012

  @thomashs @shadow I'll check if there is a problem with adding more than one email into the queue.
  
  This is currently how the code works:
  
  1) It gets your SES send rate, let's say the send rate is 5
  2) The script will add 5 emails to the queue
  3) If 5 emails are detected to have been added, it'll start sending all 5 emails at once (in parallel) to SES
  4) These 5 emails will be deleted from the queue, any errors sending will be recording
  5) 'recipients' count will be incremented based on how many was sent
  6) Step 2 to 6 will be repeated
  
  I'm not sure how more than one email is added to the queue as I have not encountered this when testing. But to be sure, I'll put a conditional statement to make doubly sure it doesn't get added more than one if one already exist in the queue. I'll also release an update immediately after I have done this.
  
  Thanks @thomashs @shadow.
• Ben

  December 2012

  I've just pushed out a new version, 1.1.3, in view of this.
  
  From the change log:
  
  Changed queuing method. Each email record in the queue will be marked as sent when sent instead of deleting the email record. The entire queue for a particular campaign will only be deleted once all emails are sent. Also added a conditional statement to check if an email already exist in queue before adding. These measures are to doubly ensure no duplicates will be sent in all cases.
  
  Get the update here → http://sendy.co/get-updated
• thomashs

  December 2012

  Ok! Let's test it!
• thomashs

  December 2012

  Just found out that sendy spent the night filling the queue table. There are millions of records again... and it sent for 100.000 contacts again.. @Ben - you have to write unit tests for your app, otherwise you are not gonna be able to grow your business with success!
• Ben

  December 2012

  Hey Thomás, did you send another campaign after this thread?
• Ben

  December 2012

  Anyways, the new update will prevent the same email from added to the queue more than once, even though I have not encountered an email being added more than once to the queue, much less millions of records.
  
  This update uses a different method of handling the queue and it's completely fail proof no matter what server environment you're on or whatsoever.
  
  As this thread is created with regards to 1.1.2, I'll now close this thread. If upgrading to 1.1.3 still gives you the same issue, either create a new thread or email me.
  
  Thanks!