Jeff Davis' SOA Ruminations

This Week in Tech – FLOSS Interview

Randal Schwartz and Leo Laporte recently interviewed me for their FLOSS (free/libre/open source software) Weekly program, which is part of TWIT (this week in tech). It was a real thrill to spend some time with these industry luminaries. You can listen to the interview here.

We discuss my book, as well as other trends in the open source community.

Manning Special Promotion

Manning Publications is offering a special promotion on my book “Open Source SOA” — buy now and save $20. This is a limited time offer, so move fast!

Experiences in rebuilding my blog server on EC2 and EBS

My blog, for the past couple of years, has been run Amazon’s EC2 environment, using a pretty old version of Ubuntu. I recently decided to upgrade my server to the Jaunty (Ubuntu 9.0.4). At the same time, I figured I’d try out Amazon’s Elastic Block Storage (EBS) for persistent storage of the database and web files. EBS addresses one of the biggest original challenges of using EC2, namely, when an EC2 crashes or is taken down, all your data is gone (unless, of course, you’ve backed everything first). As a result, fairly complicated and error-prone scripts had to be setup to backup your instance data nightly to Amazon’s S3. This always seemed problematic.

Now, with EBS, you can simply mount a persistent storage volume to your EC2 instance, and it’s treated as thought it is a regular native filesystem. Thus, if you take down your EC2 instance, your EBS data remains redundantly persisted on Amazon’s network, and can be remounted on a different EC2 instance. This turned out to be a lot easier than I had envisioned. I setup MySQL so that all of it’s data resides on an EBS device following these instructions. I also setup the Wordpress files to reside on the persistent device. In all, this all was very simple, and generally amounted to simply setting up the proper entries in fstab, such as:

/dev/sdh /vol xfs noatime 0 0
/vol/etc/mysql /etc/mysql none bind
/vol/log/mysql /var/log/mysql none bind
/vol/lib/mysql /var/lib/mysql none bind
/vol/www/blog /var/www/blog none bind
/vol/www/media /var/www/media none bind

Not being a hard-core Linux guru (more of a hacker here), I had to play around with things a bit, such as figuring out that I had to create the directories such as /var/www/blog even though it was being assigned to mount at /vol/www/blog (which is on the mounted persistent storage device).

After then installing ddclient to manage the dynamic DNS, I created a new AMI instance/bundle, and shutdown the EC2 instance. I then launched the new AMI instance, mounted the filesystem, and it all worked. Well, nearly, as I did find that I couldn’t start MySQL. It complained about the password for the user “debian-sys-maint”, which is assigned in the /etc/mysql/debian.cnf file. Apparently when I had setup MySQL originally, it used the password dynamically assigned to that account when I started up the AMI instance from alestic. When I started up this new instance, a different password was assigned that user. Fortunately, when I had setup mysql, I had created a new “wordpress” user, and I simply specified that username/password in the debian.cnf file. That worked, and mysql started like a charm.

I also configured the new instance to use the Amazon CloudWatch monitor. Here’s a snapshot of some of the monitoring features it includes:

I’m not convinced yet that CloudWatch is worth the small additional amount charge for the service, especially compared with free tools such as Nagios or Hyperic, but it does have some unique features, like the ability to spawn off new instances based on load level etc.

I continue to be amazed at the level of innovation coming out of Amazon. Maybe in a few weeks I’ll try using their new MySQL-based Relational Database Service (RDS). That looks pretty interesting.

SOA Reference Architecture

Recently the Open Group, which is an 300 member industry consortia devoted to the establishment of open standards, developed what they term as a SOA Reference Architecture. It’s a white paper that describes a blueprint for creating and/or evaluating a SOA architecture. I highly recommend it, and some of the things it touches upon are:

• Principles and benefits of SOA.
• 9 Conceptual Layers of SOA architecture, such as the role of service components, governance, business processes etc.
• Best practices for integrating middleware such as ESBs.

It’s an excellent document, and well worth reading the 30 or so pages.

Screencast Demo of Drools Flow with Apache Tuscany SCA

I’ve created a screencast that demonstrates how Drools Flow, which provides BPM-style workflow features, can be combined with Apache Tuscany SCA to create a very lightweight yet powerful dataflow orchestration solution.

Instructions for setting up the Eclipse project so that you can run the demo locally can be found at: http://d31rc4v2hihjuu.cloudfront.net/README.txt. The Eclipse project itself can be found at: http://d31rc4v2hihjuu.cloudfront.net/RuleFlowDemo.zip (the Eclipse project is rather sizable at 120mb since it includes all required libraries).

Real-time R&D enabled by the power of complex event processing

A week or so back, the WSJ Journal featured an in-depth article (http://tiny.cc/v9cAH) on the new face of innovation. Central to the article was the role that technology can now play in speeding and transforming the pace of innovation. While much has been written about how the web can be used to quickly determine how customers respond to a new idea, such rapid turn-around can now be accomplished on the retail floor, or wherever your touch-points are with your customer. How is this possible?

The authors cite the use of “digital systems” which collate and filter large amounts of data in real-time. So, when Wal-Mart rolls out a new product display, real-time statistics can be gleaned that determine how successful it is. While not mentioning the underlying technology, clearly what empowers such capabilities is complex event process (CEP), which is sometimes synonymous referred to as event stream processing (ESP). Using CEP, thousands, of even millions of events, can be analyzed through patterns or filters. Thus, using CEP, small experiments can be frequently run to determine their efficacy, without having to resort to the latent “after-the-fact” reporting that data warehouses use to provide. Greg Linden is quoted in the article as pointing out that “Constant, continuous, ubiquitous experimentation is the most important thing”. I call the use of CEP coupled with such rapid experimentation as ‘real-time R&D”.

While the companies that are cited in the article tend to be Fortune 500 enterprise customers, CEP doesn’t have to be limited to only those with deep pockets and a large IT organization. In my book, I cover an open source CEP implementation called Esper (http://esper.codehaus.org/). It’s surprisingly simple to use-and-learn, but the payback can be immense. Recently, I’ve also looked at the CEP capabilities now offered through JBoss Drools called Fusion. While I haven’t had ample time to fully kick it’s tires, it appears to offer similar capabilities to Esper (though I prefer Esper’s EPL language constructs, which more resemble SQL), and has the added benefit of being married to a rules engine — after all, you want to make decisions based upon the outcome of your pattern filters.

For anyone not yet versed on what CEP can provide, I’d recommend giving Esper or Fusion a try, and of course, my book provides a lot more coverage on the concepts and features typically found in a CEP engine.

JavaRanch Open Source SOA Giveaway

From JavaRanch’s Promotion:

We are thrilled to have Jeff Davis on the ranch to promote the book
“Open Source SOA”.

The promotion will be held in the Web Services forum which can be
found here:

http://www.coderanch.com/forums/f-51/Web-Services

Participate in this week’s giveaway by asking Jeff Davis a
question or two and you may win a copy of the book!

The giveaway starts on Tuesday, July 28th 2009.
The drawing will be held on Friday, July 31st 2009.

For more info or if you’re plumb tired of getting this info, visit:
http://faq.javaranch.com/view?BookPromotions

Interesting article about the Mozilla Foundation

The NY Times had an interesting article in today’s business section talking about the challenges Mozilla faces against the deep pocketed Google and Microsoft. Mozilla is really an amazing open source success story, and its really because of the large community of users who chip into help (coding, testing, writing extensions etc).

Book review and interview from InfoQ

InfoQ did a book review of “Open Source SOA” and asked some very good questions during an interview I had with them. See: http://www.infoq.com/articles/open-source-soa-davis.

Articles & Interviews

JavaLobby conducted an interview with me regarding the book. I thought they posed some excellent questions.

Within the next week or so, I am going to try and update my source code to reflect all the recent updates in the various packages the book covers. New versions of jBPM and Drools have been released, and a small Update to Tuscany. I will keep the existing source project in tact so that it will be consistent with the book, and create a new project with the updated libraries.