poddery.com is heavily loaded

the pagespeed should automatically handle your expires and stuff gracefully. So you shouldn't have to do anything to the assets, etc.

If it doesn't (test the pages after setting up spdy and pagespeed) just set expires max for everything that is not dynamic - css, js, txt, images, whatever. only html, php and xml etc should have no or short expiry. (I don't think you guys have any php here - ruby?)

Also, after using the pages a bit, test

curl -I https://poddery.com

You should be able to see a cache HIT header (or it may take some setting up of cache)

Another area to check is in your nginx.conf toward the top itself should be a line for worker_processes

Pravin said you have six cores, so the line should be:

worker_processes 6;

This allows you to spawn threads on all cores - should not be a huge deal, but if you've got a number smaller than 4, raising it might boost performance too.

Just came across podupti.me and saw that the poddery.com user count is less than 5000. Confirmed the same with poddery.com statistics. So, the user deletion feature of v5 seems to be enabled, deleting inactive users, causing the decrease in user count.

Presently, mysql is the most resource intensive process running. If we could migrate to Postgres, we could enable user registrations again. Last time we failed, possibly due to some error in mysql tables. Anyone have some spare time to give it a try again?

No idea on Postgres, but last I checked, the caching was next to nil - if at all it was happening. This would be a big problem. Static resources should be expires-max, and several other things. If nginx is left so unconfigured, chances are MySQL can also probably do with some optimizing - use some tuner script. The biggest server will still run out of resources if you aren't using them optimally. Poddery did not seem to be a very busy site to justify such heavy use. Any data on the number of posts/database queries, etc involved?

Vidyut

Social Media: Twitter ( https://twitter.com/Vidyut ) Facebook ( https://facebook.com/theVidyut ) Google+ ( https://google.com/+AamJanata ) Diaspora ( https://poddery..com/i/62f15bc003f0 )

Blogs: Intellectual Anarchy ( https://aamjanata.com ) || personal ( https://vidyut.info ) || Nisarga ( https://nisarga.info ) || tech ( https://vidyut.net ) || Homeschooling ( https://homeschoolingindia.in ) || Fek Le ( https://fekle.in )

Presently, the nginx load is negligible, but adding some caching and tweaking the configurations will be great. In the case of mysql, we have tuned it according to recommendations from mysqltuner (not sure about the name, but was a pearl script).

MySQL database size should be somewhere around 15GB now. The last backup dump around 3 months back, if I remember correctly, was around 12GB.

Sidekiq is the next high resource consuming process, there are 25 concurrent threads of it running, most of them idle almost all the time, as I have been monitoring them for almost 3 weeks now.

I didn't get much time to have a look at the disk io. Since mysql is causing most of the load, it might be related to disk io too, as we are still on RAID HDD.

I haven't dealt with mysql databases of this size in production elsewhere (Worked on an 8GB MSSQL and that was terrible) and not much idea about the structures in use by diaspora. So, no clear idea about how performance is affected. But have heard of many pods migrating away to PgSQL and had high performance gains.

The user deletion process is heavily resource intensive, causing high load at times.

Presently, users who haven't had any activity for 730 days are removed, after sending a mail and waiting for 30 days. Not sure when this was enabled. Might have happened in any updates around 2 months back as the deletion jobs are processed now. Looking at the scheduled jobs in sidekiq, expecting around 1000 users more to be deleted, before things get back to a stable state. At the moment of making this post, total user count is 4709, which was once above 8000.