Sendy still sending more than once to same subscriber
Ben. Looks like sendy still doesn't work properly. Today it sent more than one time to lots of subscribers again! I checked the recipients count and stil over the list count. My guess is that somehow, your script of filling the queue table before the campaign is considered good to go is not finishing before another cron call is made. So it fills the queue forever... As I said before, my queue table was going over millions of records with a sigle campaign with 15.000 subscribers. That's 2 problems: a timeout issue, that application logic should handle.. and a logic issue, to avoid sending to same contact twice (or more!!). My suggestion is: write a PYTHON or RUBY daemon that handles the sending queue, either to insert the email list into the queue and actually send them..
I'm running sendy on a Amazon EC2 micro instance. Disk space is ok.
Which variables, like timeout setting or other, do I have to check?
I'm running sendy on a Amazon EC2 micro instance. Disk space is ok.
Which variables, like timeout setting or other, do I have to check?
This discussion has been closed.
Comments
First of all, I want to say, thanks for replying and helping others out. I saw your replies on other threads.
Right after reading this thread, I sent a test email to 10,000 subscribers. I am using EC2's micro instance (same as you), on Apache / Linux. If you're wondering how I did a test send to so many emails, I set one of my email address as 'catch all', then created a list with emails like, test1@email.com, test2@email.com and so on until test10000@email.com.
Sendy sent successfully to 10,000 emails:
Amazon SES metrics says 10,000 emails were sent, no more and no less:
In 1.1.3.1, I've already made doubly sure no emails are added to the queue, twice.
1. The queue table is checked for existing email before adding an email to the queue
2. When an email is sent, it's marked as 'sent' as a second safety precautionary check so that this status will be checked against if the script happens to want to send the same email again for whatever reason it may be
As you can see I was able to send 10,000 emails using the same setup as you (EC2 micro instance) and using the latest Sendy version 1.1.3.1, there should be no issue with Sendy.
What I can suggest to you now is this:
1. Make sure you don't have any currently sending campaigns
2. Delete everything in your 'queue' table by uploading this script to the root of your installation and run it
3. Re-upload all files from version 1.1.3.1 to your installation (excluding /includes/config.php and /uploads/ folder)
4. Make sure you're not using any subversion software to upgrade Sendy as this proves to be problematic from experiences of some users
5. Make sure you're not using any caching software on your server as well
6. Also, make sure you only have one cron job executing the scheduled.php script and the timing is "every 5 mins" like so →
*/5 * * * * php /path/to/scheduled.php > /dev/null 2>&1For points 4, 5 and 6, I just want to make sure.
Which variables, like timeout setting or other, do I have to check?
Sendy automatically takes care of timeouts by automatically resuming sending if your server times out the sending script. This was highly requested by many previously and was introduced in version 1.1.1.
I deleted your previous thread and bring it here so I can answer you in one post.
Since having that queue issue, I rolled back to 1.1.1.3 which is stable and working normally for me. I saw you rolled out 1.1.3.1 last night. If I download that install, are ALL files in that release updated so that I can go from 1.1.1.3 to 1.1.3.1 without upgrading sequentially with the other several release in between?
Yes, you don't have to update sequentially. Just download 1.1.3.1. Then upload and replace everything on your server (except /includes/config.php and /uploads/ folder).
FWIW, I am having the same issue and I rolled back to sendy 1.1.1.3 as the last working release where this isn't an issue -- and dropped the queue table before rolling back.
Your issue as I previously understood was that your database's size increased by several MBs when using 1.1.3. This is resolved in 1.1.3.1 where the
query_strthat holds the encrypted email message is emptied each time an email is sent. This reduces your database size dramatically and increases sending speed at the time of sending.And, yes the queue table will be filled up with rows during sending. This is normal. They will not be deleted until all emails are sent unlike previous versions where each row is deleted after each email is sent. This is a precautionary feature (in response to @thomashs) so that the script can check the
sentcolumn to see if an email has already been sent.In 1.1.3, the
query_strcolumn that holds the encrypted email message wasn't emptied, hence your database size grew by a lot.1.1.3.1 empties the
query_strcolumn for every email marked as 'sent'. So the database size will not grow dramatically and as a result increases sending speed at the time of sending.I just sent a test email to 26,376 emails.
Here's what Amazon SES metrics displays:
Note that I've sent 10,000 emails previously for the previous test. If you look at the graph, it sent around 26000++ emails starting from 10,000 onwards. The exact quota number is not updated yet and emails are still trickling in from Amazon.
I've received every single email in my 'catch-all' gmail account. I waited a few hours to make sure no more emails are coming in. Here's the result:
Amazon SES console also shows the correct number:
Before 2nd test was sent:
After 2nd test was sent:
The difference between the 2 figures is 26,379. The 2nd test was to 26,376 emails. So between 26,379 and 26,376 there are 3 extra emails. 1 email was for the 'sending complete email notification' and 2 emails were for 2 autoresponder emails sent to 2 users who signed up for our newsletter on our website. The final figure works out to be exactly what was sent by Sendy for the 2nd test.
Sending is as accurate as it can get. It sends nothing more and nothing less as far as my test goes.
Then started the instance again, started Apache and mySQL. Auto resume kicks in and eventually finished sending.
I did the test yesterday and I have received all 15,000 emails, not any more or less.
Here's a screenshot of all 15,000 emails received:
But I never had to click resend, and at the end everything went well, without any duplicates :-)
Now I've got to find a new host...