Skip to content

Look into database usage for sending optimization

kraighkraigh
edited February 2013 in Suggestions

We have been struggling to get decent performance with Sendy (1-2 emails per second when Amazon send rate is 90/sec). After moving from GoDaddy shared Linux to a Amazon EC2 micro instance, we saw no performance increase (but database was still on a dedicated server at GoDaddy). After converting the EC2 instance to a large instance (m1.large), still no performance improvement. Then , went back to the micro instance and moved the database to the same server - huge performance improvement (bursts of 11 emails/sec with average of 5/sec over 90K subscriber list). It appears that the database is a large part for performance picture.

Ben, please review database usage, indexes, etc. I think there are big gains to be made there.

Comments

  • BenBen

    Hi @kraigh,

    If you host Sendy's files on EC2 but the database on GoDaddy, there will no doubt be latency as there are now 3 interconnecting servers (EC2, SES and GoDaddy) instead of 2 (EC2 and SES). Keeping everything together will make things much faster.

    Though, rest assured I will continue my quest to optimize sending and look into database usage.

    Thanks for your feedback.

    Ben

  • ErgunErgun

    Hi,
    I have been using Sendy for almost 3 months now where I started with a subscriber list of 77K and am now at 400+K.
    I am having the same performance issues. What I am doing right now is:
    1. I'm hosting sendy on EC2 Amazon Linux micro instance when idle.
    2. Whenever I will start emailing I change the instance type to m3.xlarge (with modifications to my.cnf file and php .htaccess with 4.096 K memory)
    3. Scale down to micro instance when send is complete

    This way I get when sending to smaller lists of 30K to 40K an average of 25-30mails/sec. But when i send to a single list of 200+K, the average speed drops to almost 10 mails per sec.

    I believe too that there is room for optimization. This is an important issue as one will never be able to use up the full quota of 1.000.000/day on a single day.

  • BenBen

    Starting from 1.1.5 released recently, indexes will be added to your database if you upgrade to this version. This should speed up database reading speed. I'll continue to see what else I can do.

  • ErgunErgun

    I have upgraded to 1.1.5.2. I have next large emailing on Thursday. I will report back the sending performance. Hope there are noticeable gains.
    Is there a way to run tables from memory? This too should increase speed.

  • ErgunErgun

    I promised to share the statistics on version 1.1.5.2 last week. Today I have sent to 462.543 subscribers on EC2 m3.xlarge instance. The overall performance was 20.72 mails/sec which is a doublefold performance increase.
    Great work, Ben thanks a lot. Hope that the performance will keep getting better and better with every update. Target is 90mails / sec. :)))

  • ErgunErgun

    There is a small issue I d like to mention: when sending the email, the progress icon refreshes all the time, and occasionally displays database connection lost error.

  • BenBen

    Thanks Ergun, glad to hear performance doubled for you. Another user also feedback on the doublefold increase in sending speed.

    when sending the email, the progress icon refreshes all the time

    You're on version 1.1.5.2, upgrade to the latest 1.1.5.4 to fix this issue.

    Thanks.

    Ben

This discussion has been closed.

© Sendy.co 2024