Hosting Sites with Heavy Traffic on Amazon EC2

I mentioned previously that I’d been looking at Amazon EC2 as a solution to host a site that underwent heavy bursts of traffic for only a couple of hours a week. The expected traffic bursts were 10-15k visitors in a 2 hour period and unfortunately a small EC2 instance just wasn’t enough. The next trial was with a large EC2 instance, with Apache tweaked to allow more simultaneous connections. Results went better than first time but I ended up having to bounce Apache a couple of times during the time frame to get it back up and running.

So fast forward to this week and it was time to try again. This time I went with an Extra Large instance running a stock 64-bit Ubuntu 8.04 and Lighttpd instead of Apache. And this time there were no problems. For the full 2 hours the site remained up and responsive, and because I could time the instance to come up just before the additional cost was negligible.

To give you an idea of the kind of traffic it had to deal with, the site took in just over 25,000 unique visitors in a 2 hour time frame. Glad to get the problem solved and now I know what to do next time I need some heavy traffic handled.

More on Hosting with Amazon EC2

After moving the hosting of oceanseventy to Amazon EC2 before departing for pastures new I was fairly keen to try it out again on a site that was more than a few hundred hits a day.  I recently had call to host a single static page that would get hit about 13,000 times within the space of a couple of hours – but do very little for the rest of the time.

So I set up a small instance running on EC2, running Ubuntu 8.04 and tweaked Apache2 a little to increase the MaxClients setting.

Unfortunately not everything went to plan.  While the server itself was rock solid the response times from Apache grew very quickly to 20+ seconds.  So I’m trying to figure out what the next plan of attack is.  Do I set up a load balancer on EC2 and share the load with a number of instances, or do I go with a larger instance for the duration of the traffic spike, upgrading to a large or extra large instance.  Are there some settings I should be tweaking on Apache?  Should I be trying out an alternate webserver like Lighttpd?

Or maybe I give up and get back to the day job  Any advice appreciated.

The Cost of Hosting Websites on Amazon EC2

As mentioned previously once of the last few things I did in my previous position at ocean70 was to migrate their existing dedicated server over from a 2 year old Fasthosts dedicated box to an Amazon EC2 instance.  That was a little over a month ago now so I thought it would be interesting to take a look at the cost for the set up in comparison to the dedicated box used previously.

To put things in perspective the box we were replacing was a 2 and a half year old Fasthosts DS 400 running Fedora 6 clocking in at £69 + vat a month (£81 in total).  In the box was a 2.8ghz Pentium 4, 1gb of Ram and a 120gb IDE hard drive.  At the time this was enough for our needs – but despite regular backups and an in-house server available for failover there was always the chance that this would go down and we would be in a right mess.  Luckily that hadn’t happened and time was right to move our hosting to EC2.

One month has passed on EC2 and i wanted to take a look at how much the server has cost to run for a month.  Total charges clocked in at $125.46 (including taxes) which at todays exchange rate equates to £84.77.  Not too shabby for a vastly improved infrastructure.

To actually have the AMI running 24/7 for a month (and remember, January is a long month) clocked in at $81.84 (£55.30) while the transfer in of 23gb cost $2.30 (£1.55) and our outgoing transfer of 25gb cost $4.27 (£2.88).  To put the bandwidth costs into perspective the instance is home to about 8 or so websites receiving fairly low traffic (max 100 uniques a day each).  The only other significant cost being the 120gb EBS block which cost $15.00 (£10.13) for the month.

So thats a £5 a month increase for a system whose hardware will never date.  Can be brought back up in 5 minutes should anything go wrong.  Has virtually limitless storage facility, and on top of that is a hell of a lot faster than its predecessor.  Bargain.  Will be interested to see how it averages out over the year, especially as a few more sites are put on it.

Hosting Migration to Amazon EC2

As I mentioned previously, we (oceanseventy) decided that recently to take advantage of the slowdown around Christmas and start to migrate our hosting set up away from a Fasthosts dedicated server.

For 2 years we’d been using a Fedora 6 box which had been running fine, but it was becoming difficult to keep the OS up to date given Fedora’s habit of ignoring previous releases fairly quickly.  Other issues, like the fact that there was a single hd in the machine which could potentially die at any time and cause much pain, prompted the desire to move everything to something more robust, flexible and easy to backup.  Enter Amazon EC2.

Fedora was ditched in favour of Ubuntu 8.04 – the latest LTS release – and within an hour we had set up a small instance in the Euro availability zone.  The doc’s for EC2 are excellent and take you through the process.  An Elastic IP was attached to the instance to ensure we could maintain the ip should the server ever go down.  

We also attached a 120gb Elastic Block Store instance, mounted at /var/www to make it easy to take all data easily to another instance should the need arise.  Backups are handled with the  AMI and EC2 tools.  Nightly the instance is bundled and stored on S3 and nightly a snapshot is take of the EBS block as well.

So far the instance has been rock solid reliable and much faster than the old Fasthosts dedicated box we had.  EC2 is exactly what anyone hosting websites need at the moment – no tie-in to any physical hardware and the flexibility to scale your hardware as required.

I’ve seen a lot of criticism about hosting web sites on EC2 but almost all of the issues I’ve seen are no longer relevant as Amazon have taken steps to improve EC2 and add critical features (like persistent external IP addresses).  

Really glad I got the opportunity to implement this as my last act at oceanseventy.  I’m definitely and EC2 advocate now and hopeful the skills i’ve learned will help my in my new position at STV.  Hopefully i’ll be able to use EC2 for my startup project as well.