Experiences in rebuilding my blog server on EC2 and EBS

My blog, for the past couple of years, has been run Amazon’s EC2 environment, using a pretty old version of Ubuntu. I recently decided to upgrade my server to the Jaunty (Ubuntu 9.0.4). At the same time, I figured I’d try out Amazon’s Elastic Block Storage (EBS) for persistent storage of the database and web files. EBS addresses one of the biggest original challenges of using EC2, namely, when an EC2 crashes or is taken down, all your data is gone (unless, of course, you’ve backed everything first). As a result, fairly complicated and error-prone scripts had to be setup to backup your instance data nightly to Amazon’s S3. This always seemed problematic.

Now, with EBS, you can simply mount a persistent storage volume to your EC2 instance, and it’s treated as thought it is a regular native filesystem. Thus, if you take down your EC2 instance, your EBS data remains redundantly persisted on Amazon’s network, and can be remounted on a different EC2 instance. This turned out to be a lot easier than I had envisioned. I setup MySQL so that all of it’s data resides on an EBS device following these instructions. I also setup the Wordpress files to reside on the persistent device. In all, this all was very simple, and generally amounted to simply setting up the proper entries in fstab, such as:

/dev/sdh /vol xfs noatime 0 0
/vol/etc/mysql /etc/mysql none bind
/vol/log/mysql /var/log/mysql none bind
/vol/lib/mysql /var/lib/mysql none bind
/vol/www/blog /var/www/blog none bind
/vol/www/media /var/www/media none bind

Not being a hard-core Linux guru (more of a hacker here), I had to play around with things a bit, such as figuring out that I had to create the directories such as /var/www/blog even though it was being assigned to mount at /vol/www/blog (which is on the mounted persistent storage device).

After then installing ddclient to manage the dynamic DNS, I created a new AMI instance/bundle, and shutdown the EC2 instance. I then launched the new AMI instance, mounted the filesystem, and it all worked. Well, nearly, as I did find that I couldn’t start MySQL. It complained about the password for the user “debian-sys-maint”, which is assigned in the /etc/mysql/debian.cnf file. Apparently when I had setup MySQL originally, it used the password dynamically assigned to that account when I started up the AMI instance from alestic. When I started up this new instance, a different password was assigned that user. Fortunately, when I had setup mysql, I had created a new “wordpress” user, and I simply specified that username/password in the debian.cnf file. That worked, and mysql started like a charm.

I also configured the new instance to use the Amazon CloudWatch monitor. Here’s a snapshot of some of the monitoring features it includes:

I’m not convinced yet that CloudWatch is worth the small additional amount charge for the service, especially compared with free tools such as Nagios or Hyperic, but it does have some unique features, like the ability to spawn off new instances based on load level etc.

I continue to be amazed at the level of innovation coming out of Amazon. Maybe in a few weeks I’ll try using their new MySQL-based Relational Database Service (RDS). That looks pretty interesting.

Sun's Disappointing Cloud Announcement

Sun today announced some specifics about its new cloud initiative, and I must say, I was pretty disappointed, particularly in the details about their API. Apparently they are supporting only “RESTful” APIs, thus effectively limiting their audience to non-enterprise customers. Why do I say this? If you’re using products like BizTalk, or any of the multitude of ESBs out there, you know that working with SOAP-based APIs is a FAR more straightforward proposition. In many cases, the tooling will simply suck in a WSDL, and a graphical mapper makes developing the interface touch points very simply. Of course, testing is far easier as well using SOAP-based APIs, since open source tools like soapUI are so elegant and simple to work with. Not to mention, of course, if you’re using any BPEL for orchestration, you’re out of luck with REST.

Quite frankly, it baffles the hell out of me why anyone now would develop a greenfield API and not plan on supporting a multitude of protocols “out-of-the-box”. For example, if you use Apache Tuscany, you get automatic support for REST, SOAP, JMS, JSON, RSS without having to write any protocol-specific code whatsoever. You develop your components using any multitude of languages, and then declaratively, in XML, define how you want to expose them as services.

Instead, what Sun has done is effectively alienate a good part of the possible user community, and instead focus on non-enterprise users who use scripting languages such as PHP, Ruby and the like (nothing against scripting languages, as I use them all the time myself, such as Groovy). I just can’t envision why you wouldn’t support multiple protocols from the get-go.

I must say, Amazon has shown a lot more enterprise savviness than Sun — who’d have thunk!

If you want to learn more about Tuscany, the official website is obviously a great place to start, or simply buy my book!