Amazon EC2 as a Build and Development Environment

How many people are faced with using machines where they're not given root access, it's a headache to get additional software installed and you're required to access those systems using a slow VPN? Although that list is short, there are a number of impedances to making any real progress in there. So to short-circuit the political morass, I took a look at Amazon EC2 and so far I am quite pleased.

Just before the holidays, I took a day to dig in to Amazon EC2 and I must say that having the ability to quickly (less than 30 mins) spin up a machine image, sized to my liking where I have total control of the operating system and the software installed on it is exactly what I needed as a developer.

It took me a couple hours to walk through and understand all the steps to create an image, but it was more than worth it. I did quickly discover that the docs don't tell you that you must size the image to your liking when it's created (see the ec2-bundle-vol utility's --size option). The default image size is 1GB (which is pretty small) and the maximum size is 10GB - I now use 10GB :-). The other thing I discovered was that shutting down an image will clobber *everything* you've installed on that image. This caused me to have to recreate my image a couple times after installing a bunch of software and configuring the image to my liking.

There are certain limitations like I've mentioned above with the storage, no static IPs assigned to images, inability to hack the kernel, and other things. The storage shortcoming I'm already working around by looking into using an Amazon S3 bucket as a filesystem to easily store stuff. The other stuff is really not critical for me so far because I'm just using it as a build and development environment. Maybe if I decide to host real apps on an image someday I'll worry more about them. But for the time being, providing me an environment over which I have total control and only need SSH to access is a major leg up.

There is a cost involved but it's minimal when compared to hosting a machine in a facility. If you need a machine where you have total control and a low cost, I encourage you to take a look at Amazon EC2.

Associating PIDs with Open Sockets on MacOS X

For years I've used netstat to determine which process is holding onto an open socket. That is, until MacOS X came along. I really like MacOS X but it drives me nuts at times when I find little differences like this that I've relied upon for so long. I know that this due to my lack of familiarity to the differences between Linux and BSD. Anyway, I recently stumbled upon an option while reading the man page for lsof that made me think it might substitute for this missing functionality:

-P  This option inhibits the conversion of port numbers 
    to port names for network files. Inhibiting the 
    conversion may make lsof run a little faster. It is 
    also useful when host name lookup is not working 
    properly.

So I gave it a shot (wrapped for display):

$ lsof -P | grep 61616
java      7457 bsnyder   33u    IPv6 0x39237f8        0t0      
TCP *:61616 (LISTEN)
java      7457 bsnyder   43u    IPv6 0x6278a24        0t0      
TCP [::127.0.0.1]:55496->[::127.0.0.1]:61616 (ESTABLISHED)
java      7457 bsnyder   44u    IPv6 0x642c8c4        0t0      
TCP [::127.0.0.1]:61616->[::127.0.0.1]:55496 (ESTABLISHED)

Notice the PID in column two of the output above. This is can be incredibly useful especially when piped to other utilities (wrapped for display):

$ ps ax | grep `lsof -P | grep 61616 | awk {'print $2'} \
| uniq`
7457  p6  S+     0:09.10 
/System/Library/Frameworks/JavaVM.framework/Home/bin/java -Xmx512M 
-Dorg.apache.activemq.UseDedicatedTaskRunner=true 
-Dderby.system.home=/Users/bsnyder/apache-activemq-5.0.0.5

Voila! Now I can see what process is holding open port number 61616 (but I already knew that ;-) ).

Now, if I can just find a way to use the --forest option on the ps utility on MacOS X. Using ps -ef --forest on Linux is incredibly helpful yet unavailable on MacOS X. Anybody have any suggestions?

Apache ActiveMQ 5.0 Released

The long anticipated Apache ActiveMQ 5.0 has finally been released! Congratulations all around, a lot of hard work went into this release. Check out the changelog.

Spring 2 Book Finally Released

My latest book named Beginning Spring Framework 2 has finally been released. I coauthored this book with Thomas Van de Velde, Christian Dupuis, Sing Li, Anne Horton and Naveen Balani. Check it out ;-).

Apache ServiceMix @ JavaPolis 2007

Last week I was in Antwerp, Belgium to speak at JavaPolis 2007 about Apache ServiceMix. There were loads of good presentations and tons of people. This year JavaPolis set a new record for attends as there over 3200! And let me tell you, the place was packed! Yet another wonderful conference this year. Stephan and his team really do a fabulous job.

I got to meet many new folks and see a number of folks who I don't see that often including Dion and Bob who, oddly enough, both live on the West coast in California, and yet we all traveled half-way around the world before we saw one another. I hung out with my friend Oisín Hurley and his good friend Stephen McNamara and we had some wonderful dinners and spent some good time together.

I really enjoy Belgium and The Netherlands any time of year. It was cold and very damp but I still find that area of the world very intriguing. There is so much ornate architecture, artwork and extremely old buildings to be seen that are rich in history. This is very different from most of the U.S. because the U.S. is fairly young relatively speaking.

The Cult in Denver, CO

Chris and I saw The Cult on Monday night at the Fillmore in Denver. I've seen them numerous times over the years and I've always been a very big fan. They played a number of older songs which was cool, but they also only played for about 75 mintues. We both found this kinda odd.

The new album is called Born Into This and is quite good, both lyrically and musically. It really stabs at life today as being quite gluttonous, eluding to the fact that the world culture has lost its way. In fact, during the show they played the video below for one of their latest songs.



The video includes footage of demonstrations and police crackdown in Burma but evidently the song is about rampant consumerism. Reading the lyrics makes this a bit clearer.

It was good to see them again, but for some reason the show wasn't that long and the entire band seemed to leave the stage in anger - who knows. I still like The Cult ;-).

Sun's Appalling Behavior With OpenDS

Recently it came to my attention that Sun has exhibited yet another set of appalling behavior on one it's so-called <quote>open source projects</quote>. IMO, anything Sun has released as open source that only contains Sun folks is always suspect. This particular episode is quite egregious and really demonstrates Sun's lack of understanding of how open source works.

Without rehashing the details, I'll just provide a link to Dave Shields' blog entry about this particular occurrence which is aptly titled OpenDS: On Being Bitten By The Hand That Once Fed You. David also links to the An Open Letter to the OpenDS Community and to Sun Microsystems that details the real story.

I really feel for the guys on the project who tried to do the right thing as the situation that Sun forced on them really does suck. This is a good demonstration of how big companies ruin open source projects.