Making Your Web Server Crawl

Over the past six months I have increased the server capacity for this blog, and for my other sites, three times. The latest Media Temple upgrade happened last week but there were still moments even after the upgrade when this blog was spiking the server. When a server meets its capacity strange things happen and the “user experience” begins to slow down for everyone and that’s a Bad Thing.

Since I don’t want to spend $2,500.00 USD a year on upgrading my hosting plan again, I have made some changes here and there that you may have already noticed. Some of these changes were suggested by my web server host Media Temple while I implemented others to see how removing features would relate to increased speed. Most of these changes are WordPress 1.5.2 specific but some of the ideas expressed below might be directly applicable to any blog software or Content Management System you are running.

REMOVED FEATURES Using PHP to send mail can clog things up and this blog was processing a lot of “email” via PHP instead of a traditional mail server. Here are some of those email burdens:

1. Contact Page: I removed this feature because kiddies were using it to send me junk. If you want to contact me via this blog you may leave a comment and I will reply to your email address.

2. Subscribe to Comments: My web host suggested this was the major “feature” of this blog that was slowing everything down because email had to be sent to all subscribers via PHP and that takes a lot of server effort to make happen for a popular post. Bad people can also choke your server by subscribing to all your posts from lots of different email addresses — that drowns your server in email requests that don’t ever go anywhere. Here are some of the other Plugins I removed because they required lots of database queries and that also adds stress on your server if you have a popular blog:

3. Related Posts: I loved seeing how a post today reflected five posts from the past but I can live without seeing that information because calling up five other random posts for every post view adds cycles to the load time and I want everything here to load as fast as possible.

4. Latest Comments: I also loved this Plugin, but to call up the commenter and adding other information about the post also puts a stress load on your server that wasn’t doing me much of a good thing — especially since every other commenter was… “Mr. David W. Boles, MFA.” I don’t need a Plugin to tell me how often I comment.

5. Noteworthy: This was a fun Plugin to use but, in the end, it didn’t add much content or provide a strong enough frame to warrant its continued use.

MODIFICATIONS Here are some other changes I made to reduce sever load.

6. I lowered the default index page posts from the last 7 posts to the last 3. I liked seeing a week’s worth of posts on the main page but 3 will always load faster than 7 so — 3 wins.

7. My web host also suggested I turn off Trackbacks and Pingbacks to save server stress. I have done that to see if things speed up here. My web host also suggested I only allow comments from registered users. I may implement that in the future because it will certainly cut the Spam processing down to zero. For now, since all comments are moderated due to the current WordPress comment danger, we will leave things alone.

8. I lowered the “RSS feed request” from the last 30 posts to the last seven. That change will prevent Nasty People from overloading the server by placing tons of phony “RSS Subscribe” requests to stall my server.

STAYING FOR NOW Here are some of the features that add a bit of a toll to the server but, for now anyway, they’re staying.

9. Rotating Image Header: I like the look and feel of the header images. Loading up a new image takes time and server effort, though, so one day I may need to just pick one of the images and leave the header image static.

10. K2: I like the design of K2 as a WordPress template and I like the AJAX comments processing where the page shifts and loads as you place a new comment. That kind of rich feature adds to the PHP processing time but, for now anyway, it’s worth it to me to keep.

11. Sending a full RSS feed: I don’t like to read truncated RSS feeds where you have to click on the article in your RSS feed reader and actually visit the site to read the entire blog post. I will continue to serve full text feeds of all posts and comments here so you can continue to read this blog as you wish.

TO COME When I finally upgrade WordPress to Version 2 — and I will do that as soon as K2 comes out of beta — I will employ other changes here that will also speed up the responsiveness of this blog on its current hosting plan.

12. My philosophy is to be as bare boned as possible without being totally ugly even though I prefer speed over pretty. With these changes in place the average load time for the main index page has dropped from 73 database queries in 2.87 seconds to 46 database queries in 0.80 seconds — so you can see the speed increase is already obvious and edible — and those faster times do not consider the added massive reduction in PHP mailing stress that was relieved behind-the-scenes.

Veronica!
Exactly! There are some heartbreaking changes — like losing the Subscribe to Comments Feature — that bothers me but when there are nasty elements out there abusing your features you have to move to cut them off or you’ll end up paying for their the problems they create for the legitimate readers and commenters.
You do have to strike a delicate balance between paranoia and aesthetic.

Hi Veronica —
Yes, that’s what I understand about WordPress 2, too. There are some bloggers who feel 2 adds a lot of code bloat so that might weigh you down interactively as a BlogMaster in other ways.

That might work good for you if you can tweak 2 to your liking. You could get really fast if you just posted once-a-week and didn’t allow comments ever or if you just deleted your entire blog from the web!
😆

Don’t restrict comments to registered users only.
While I’ll probably register since this is like the “Cheers” bar where there are a bunch of great regulars, you’ll probably lose some of the great comments from people who may only drop by from time to time.
I know when I’ve seen blogs that require someone to sign up for the service without providing for a way to post anonymously, I often don’t bother to leave a comment because I don’t want to bother remembering another username and password.

Hi Chris —
I appreciate your feedback on requiring registration for commenting here.
I would be satisfied with a few dedicated “commenters” who registered even if it meant the loss of the casual commenter. I prefer one thoughtful, well, argued comment to 10 single-sentence comments.
Registration would also kill all my Spam in its tracks and get rid of the kiddie comments that I have to manually delete every day.
You can also give registered users more access and enhanced “roles” in the integration and publication of a blog so it can truly become a larger community effort.

Hi Carla!
Now that’s interesting! Does it look the same as what I was using here? Or is this something specially coded for WordPress.com that is set up to work really well with that static hosting solution?

It looks different than your Recent Comments section did. It simply lists the name and the first words of the comment (whatever fits on one line). I’m not sure about the coding. All I know is that they made it part of the sidebar widgets, which are not customizable in that you can decide the order you want them in on your WordPress.com blog.
You still can’t mess with the template and stuff, though. I know some people are still pretty bummed about that. They want to put their Flickr box or Google Ads in their sidebars. It would be pretty cool to have that, but I’m not that interested in it. Besides, with my basic knowledge of html, it’s probably best that I not have the temptation to mess with it! 😀

Interesting information David, I was contemplating the Recent Comments plugin but now will not implement it.
I was thinking about posting photos and image mainpulations I play around with on our main blog. But started a Blog Charm blog for those so as not to slow the WordPress blog down.
Mik

Hiya Mik!
Recent Comments is a glorious plugin — implementing it is a choice you have to make based on your individual setup.
If you add a page load timer in your footer like I have here in mine you can test the database queries and page load times yourself. Load a page a few times with the plugin active and then load the page a few times without the plugin active and compare the results.
Images can be tricky! Flickr has been slow lately and if a site is slow it slows down your site load times as well. I like having all my “stuff” on a sever I control because, in theory, everything will load at the same speed.
When you make “off-site” calls in your HTML your site load times are at the mercy of servers you do not control and that spells trouble!
😀
You plan with BlogCharm sounds smart and right on target!

What webhosting solution are you using? If it would require $2,500 dollars to upgrade your plan, I would venture to say “the wrong one.”
These plugins you’ve mentioned don’t bog down the server as much as you think. And in this day and age of broadband, Web 2.0 and AJAX, more and more functionality is possible without the burdens being noticeable.
Dreamhost and Media Temple are rock-solid solutions. And you’ll pay alot less for better service.

You’ve got to be kidding me. Half of the WordPress world is on Media Temple. 9rules is on Media Temple. Do you know how many queries their frontpage makes? How many plugins are required to run that site? How much custom code?
If Media Temple is making these suggestions, then apparently they don’t appreciate the money you give them. Out of all your sites, if I’m correct in assuming, this is the only one that runs on WordPress. Your other sites, are for the most part, static html?
I pay $120 a year to fully host as many domain names as I want. And Dreamhost has never told me to not use certain plugins. I understand that you want to make your site as fast as possible–your audience requires it. And while Dreamhost does have its problems, they have never made me feel less valuable as a customer. As far as I’m concerned, your site was barebones before the modifications. I just don’t get it.
Media Temple, I’m flabbergasted.

Hi David
I’ve just discovered something quite interesting about “subscribe to comments”. The code calls the PHP email function once for every person subscribed to a post. If there are several subscribers, and frequent new comments, then it will cause a lot of calls to “mail”, the PHP function.
Ideally, for each subscriber to a post a single email should be sent, with the sender set correctly and all subscribers added in the BCC: field. This will lose the “personal touch” of an email addressed directly to each subscriber, but will lighten load considerably. One call to “mail”, and the message is queued by the MTA which should be optimised to handle sending to multiple recipients.
It’s the sort of thing I could probably fix in a plugin like that, but I wonder if the authors aren’t already working on it…
Don’t know if this part will work but here’s the code in question :
foreach($subscriptions as $email) {
if (!$this->is_blocked($email->comment_author_email) && $email->comment_author_email != $comment->comment_author_email && is_email($email->comment_author_email)) {
$message_final = str_replace(‘[email]’, $email->comment_author_email, $message);
$message_final = str_replace(‘[key]’, md5($email->comment_author_email . DB_PASSWORD), $message_final);
$this->send_mail($email->comment_author_email, $subject, $message_final);
}
} // foreach subscription
-Fruey