Queue table is full to be sent but nothing going out...
Sendy Support,
I purchased yesterday and got the latest 1.1.0 setup on one of our AWS servers. I imported one of our member email CSV (blank name, email) which was approximately 65k emails. I also created a "test list" (5 emails) which was just quickly typed up versus an imported CSV.
I setup cron on the server so that we could schedule and resume items as well.
I then created a new campaign and sent it out to the "test list" to see how things worked and ensure we had everything setup correctly. Emails went through to the test list with no problems and everything was working great -- opens, link tracking, etc.
I duplicated the original campaign and selected the "member list" (65k) emails and told it to send. It processed for about 15ish minutes as the number kept increasing for "sent" records until it was officially marked as "sent". The only challenge is zero "read records" which makes no since so I started digging more --> amazon SES console show no emails sent and all the emails are sitting in the queue table.
FYI, our Amazon SES are 28 emails/sec with a quota of 100k/24hours and right now SES console states we have 99984 Remaining.
I also checked to make sure that all emails in the "member list" are "subscribed" status and they are. Why aren't they being sent? What else can I check to get them to go?
Thanks,
Jeremy
I purchased yesterday and got the latest 1.1.0 setup on one of our AWS servers. I imported one of our member email CSV (blank name, email) which was approximately 65k emails. I also created a "test list" (5 emails) which was just quickly typed up versus an imported CSV.
I setup cron on the server so that we could schedule and resume items as well.
I then created a new campaign and sent it out to the "test list" to see how things worked and ensure we had everything setup correctly. Emails went through to the test list with no problems and everything was working great -- opens, link tracking, etc.
I duplicated the original campaign and selected the "member list" (65k) emails and told it to send. It processed for about 15ish minutes as the number kept increasing for "sent" records until it was officially marked as "sent". The only challenge is zero "read records" which makes no since so I started digging more --> amazon SES console show no emails sent and all the emails are sitting in the queue table.
FYI, our Amazon SES are 28 emails/sec with a quota of 100k/24hours and right now SES console states we have 99984 Remaining.
I also checked to make sure that all emails in the "member list" are "subscribed" status and they are. Why aren't they being sent? What else can I check to get them to go?
Thanks,
Jeremy
This discussion has been closed.
Comments
Yes, the total record count in the queue table is still 65384 records and has been since the campaign is defined as "sent".
FYI, I also looked at the difference in the 2 different "lists" (member and test). All the data is the exact same and only difference is the blank name field - not null btw.
Let me know anything else you would like me to check or any other questions.
I tried to put all the code onto a different AWS server of ours to ensure that it wasn't a PHP setup since the 2 are configured differently.
The apache logs from both servers have this same error in them:
[Fri Nov 16 09:40:06 2012] [error] [client 71.204.42.71] PHP Warning: array_unique() expects parameter 1 to be array, null given in /var/www/html/sendy/report.php on line 651, referer: http://ourDomain.com/reports?i=1
[Fri Nov 16 15:05:28 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/app?i=1
[Fri Nov 16 15:06:46 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/send-to?i=1&c=6
[Fri Nov 16 15:09:12 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/send-to?i=1&c=6
[Fri Nov 16 15:09:17 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/send-to?i=1&c=6
[Fri Nov 16 15:13:26 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/app?i=1
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: Missing argument 1 for PHPMailer::IsAmazonSES(), called in /var/www/html/sendy/includes/create/test-send.php on line 111 and defined in /var/www/html/sendy/includes/helpers/class.phpmailer.php on line 419, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: Missing argument 2 for PHPMailer::IsAmazonSES(), called in /var/www/html/sendy/includes/create/test-send.php on line 111 and defined in /var/www/html/sendy/includes/helpers/class.phpmailer.php on line 419, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: Missing argument 3 for PHPMailer::IsAmazonSES(), called in /var/www/html/sendy/includes/create/test-send.php on line 111 and defined in /var/www/html/sendy/includes/helpers/class.phpmailer.php on line 419, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: simplexml_load_string(): Entity: line 1: parser error : Start tag expected, '<' not found in /var/www/html/sendy/includes/helpers/class.amazonses.php on line 83, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: simplexml_load_string(): HTTP/1.1 200 OK in /var/www/html/sendy/includes/helpers/class.amazonses.php on line 83, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:30:56 2012] [error] [client 71.204.42.71] PHP Warning: simplexml_load_string(): ^ in /var/www/html/sendy/includes/helpers/class.amazonses.php on line 83, referer: http://ourDomain.com/send-to?i=1&c=7
[Fri Nov 16 15:32:15 2012] [error] [client 71.204.42.71] PHP Warning: Division by zero in /var/www/html/sendy/app.php on line 120, referer: http://ourDomain.com/send-to?i=1&c=7
Let me know of anything else I can provide to help get this working - got lots of emails to send out ;o)
Thanks!
I have heard feedback from other users who were able to send successfully on EC2 http://sendy.co/discussion/227/holy-moly-sendy-is-fast#Item_4 May be some configuration differences.
Are you running PHP on safe mode?
I wondered about the PHP setup as well and saw the other forum post about 'safe mode'. I am not running in safe mode on either server tested. This is also why I tested it on 2 different servers. I am REALLY thinking this has something to do with the 2 different list sizes -- the "test list" with only 5 emails sends no problem. The "member list" with 65,000+ emails in it does not send at all.
FYI, I also tried to delete all the people in "member list" and uploaded it again with a "name" input in it and that didn't make a difference. I will divide it up into several smaller list for curious to see what happens and report back.
Was there a different setting or database entry for installations previous to 1.1.0 that weren't in the latest?
Thanks!
Here's how it works. The 1.1.0 sends your emails using 'multi-threading' if the number of emails you're sending to is more than your SES send rate. I'm sure your SES send rate is more than 5. So when you sent to 5 emails, 'multi-threading' is not used to send to these 5 emails. The sending code in previous versions of Sendy is used to send these 5 emails instead.
When you sent to 65k, multi-threading kicks in. Sendy first gets your SES send rate. Let's say your send rate is 28 emails per second. The script will insert each email into the 'queue' until it reaches 28 records, then the next iteration in the loop will blast these 28 emails from the queue to Amazon using non blocking CURL. Once these 28 emails are transferred to Amazon, they will be deleted from the queue. The cycle then continues.
It looks like the 'non blocking CURL' portion used to send your emails to Amazon is not working on your installation. That's why the queue isn't emptied.
Can you try sending a test email to 28 emails (or whatever number is your SES send rate) so that multi-threading will be used to send this email and verify that it works? You may not have 28 emails to test (maybe you do), but here's what you can do if you don't. Set one of your emails to 'catch all'. Then create a test list in Sendy with test1@domain.com, test2@domain.com and so on.
Interesting thing is both campaigns with the smaller lists sent with no problems -- tried 1k list and sent and then the 10k list and it sent too. The queue table grew by approximately 11k and then went back down to a row count of 65379 which is the remaining from the original member list.
So I am curious based off your explanation of the "multi-threading" why would the 1k list and the 10k list both send correctly but the 65k list did not send and is still sitting in the queue table?
New thoughts?
Thanks.
Perhaps those warnings (that you pasted above) caused sending to fail. Another user also reported these warnings. 1.1.1 fixes these. Moreover the code was further optimized. Though, some users didn't have any problems with 1.1.0.
Anyways 1.1.1 should be more stable. You should clear out the 65379 records remaining in your queue table (using DELETE FROM queue) and then send to the rest.