WordPress uploads and load balancing

I am currently working on a project building out a load balanced WordPress install with multiple web servers and I came across an issue that I think I have found an innovative way to address.  Recently at WordCamp San Francisco, Mark Jaquith in his session Scaling, Servers, and Deploys — Oh My! stated that his preferred way to handle uploads in a load balanced WordPress environment is to use a single server for WordPress admin and media uploads and then use some type of task to push the media out to the other servers.  I think this is great for most load balanced WordPress environments since the load from admins is minimal compared to end users.

However, what happens when you give your end users the capability to upload media?  You certainly don’t want to move your end users to a single server — that defeats the purpose of having a load balancer.

In my case, the project I am working on is being built on RackSpace cloud servers and is using RackSpace cloud files for CDN distribution of static images.  The thought occurred to me that there had to be a way for my cloud servers to somehow share uploaded files.  After googling around I came across CloudFuse.  CloudFuse essentially allows me to mount RackSpace cloud files as a directory on my web servers.  Each web server has a directory that mounts the cloud files and WordPress’ upload path is set to that directory.  Multiple web servers seem to play nicely with uploading files to the directory, though I have not tested it under an type of heavy load.

There is also another benefit to this solution.  Since cloud files include CDN distribution, the uploaded files can then be served using the CDN which circumvents two things:

  1. Having to push the uploaded images to multiple servers and/or a CDN.
  2. The CloudFuse mount only has to be used for write operations.

Admittedly this solution only works if you are using RackSpace, though I believe you could pull off something similar with other CDNs.

About these ads

4 thoughts on “WordPress uploads and load balancing

  1. Any news on how well this worked? An issues with cloud fuse / cloud files stability?

    We are looking at doing it on a site that gets a fair number of uploads (call its 10k+ files per day), but wanted to see if anyone had managed to use this technique in production for an extended period.

    • We have been doing this in production for about 7 months and it has worked flawlessly for us. Performance wise it seems slightly slower than a regular file write, but not really a noticeable dIfference from a user’s perspective. I know we are nowhere near 10k uploads a day though; however assuming that load is split over multiple servers I believe it should scale well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s