04 February 2010

Using the Spring JmsTemplate to Send JMS Messages



Recently I stumbled upon a number of places in the some docs and mailing lists where claims are made that the Spring JmsTemplate is full of anti-patterns, is horribly inefficient and shouldn't be used. Well I'm here to debunk these erroneous claims by pointing out a class in the Spring Framework that was overlooked entirely.

The Spring JmsTemplate is a convenience class for sending and receiving JMS messages in a synchronous manner. The JmsTemplate was originally designed to be used with a J2EE container where the container provides the necessary pooling of the JMS resources (i.e., connections, consumers and producers). Such requirements came from the EJB spec. But when developers began using the JmsTemplate outside of J2EE containers, and because some JMS providers do not offer caching/pooling of JMS resources, a different solution was necessary. Enter the Spring CachingConnectionFactory.

The CachingConnectionFactory is meant to wrap a JMS provider's connection to provide caching of sessions, connections and producers as well as automatic connection recovery. By default, it uses a single session to create many connections and this model works very well with most MOMs. But if you need to scale further, you can also specify the number of sessions to cache using the sessionCacheSize property.

Below is a snippet from a Spring app context that demonstrates the configuration for the CachingConnectionFactory


...
<!-- A connection to ActiveMQ -->
<bean id="amqConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory"
p:brokerURL='tcp://localhost:61616" />

<!-- A cached connection to wrap the ActiveMQ connection -->
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory"
p:targetConnectionFactory-ref="amqConnectionFactory"
p:sessionCacheSize="10" />

<!-- A destination in ActiveMQ -->
<bean id="destination"
class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="FOO.TEST" />
</bean>

<!-- A JmsTemplate instance that uses the cached connection and destination -->
<bean id="producerTemplate"
class="org.springframework.jms.core.JmsTemplate"
p:connectionFactory-ref="cachedConnectionFactory"
p:defaultDestination-ref="destination" />
...


As you can see, the configuration for the CachingConnectionFactory along with the JmsTemplate is quite simple. Furthermore, these two classes are also both in the org.springframework.jms package path so they're both included in the spring-jms jar file making their use even easier.

The only thing left to do is utilize the jmsTemplate bean in your Java code to actually send a message. This is shown below:


public class SimpleMessageProducer {

private static final Logger LOG = Logger.getLogger(SimpleMessageProducer.class);

@Autowired
protected JmsTemplate jmsTemplate;

protected int numberOfMessages = 100;

public void sendMessages() throws JMSException {
StringBuilder payload = null;

for (int i = 0; i < numberOfMessages; ++i) {

payload = new StringBuilder();
payload.append("Message [").append(i).append("] sent at: ").append(new Date());

jmsTemplate.send(new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
TextMessage message = session.createTextMessage(payload.toString());
message.setIntProperty("messageCount", i);
LOG.info("Sending message number [" + i + "]");
return message;
}
});
}
}
}


The SimpleMessageProducer class above demonstrates the use of Spring autowiring to resolve the relationship between the jmsTemplate property and the producerTemplate in the app context further above. Then an anonymous MessageCreator instance is used to actually create a message for the jmsTemplate to send.

The JmsTemplate and the CachingConnectionFactory are both very widely used in businesses of all sizes throughout the world. Coupled with one of the Spring message listener containers, they provide an ideal solution.

I'll elaborate on message consumption using the Spring DefaultMessageListenerContainer and the SimpleMessageListenerContainer in another blog post.

21 January 2010

How to Use Automatic Failover In an ActiveMQ Network of Brokers

Last week I tested a new feature in ActiveMQ 5.3.0 to support automatic failover/reconnect in a network of brokers. Besides adding this information to the ActiveMQ book, one person also suggested that I also post it on my blog for easier access, so here you go!

Folks familiar with ActiveMQ already know that a network of brokers allows many broker instances to be networked for massive scalability. Prior to the addition of this feature in ActiveMQ 5.3, if one of the brokers in the network went down, reestablishing a connection with that broker when it comes back up is a manual process wrought with difficulty. By adding support for failover to the network of brokers, any broker in the network can come and go at will without any manual intervention. A very powerful feature, indeed. Although this post is long, the outcome of the testing is well worth it.



The first thing to note is the topology for the network of brokers. I used a network of three brokers named amq1, amq2 and amq3. The attached diagram explains the topology, including the consumers and producers. amq1 and amq2 are stand alone with no network connector. amq3 defines a network connector with failover to amq1 and amq2. Consumers exist on amq1 and amq2. Producer will connect to amq3. To start with, I have only configured a uni-directional network connector in amq3. Later I will change the configuration for a bi-directional network connector.

Thanks to the ability to upload any file to Google Docs this week, you can download the configuration files for the three brokers.

The next thing to do is outline the steps I used to test out this feature. These steps were performed on Mac OS X (Unix) but could easily be adapted for Windoze. Below are those steps:

1) Open six terminal windows as defined below:
1a) Terminal 1 = cd into the amq1 dir
1b) Terminal 2 = cd into the amq2 dir
1c) Terminal 3 = cd into the amq3 dir
1d) Terminal 4 = cd into the amq1/example dir
1e) Terminal 5 = cd into the amq1/example dir
1f) Terminal 6 = cd into the amq1/example dir

2) Terminal 1: start up amq1 (./bin/activemq)
3) Terminal 2: start up amq2 (./bin/activemq)
4) Terminal 3: start up amq3 (./bin/activemq)

Thanks to the configuration of the ActiveMQ logging interceptor, you should see that amq3 makes a network connection to either amq1 or amq2. For the rest of these steps, let's assume that amq3 connected to amq1.

5) Terminal 4: start up a consumer on amq1 (ant consumer -Durl=tcp://0.0.0.0:61616)
6) Terminal 5: start up a consumer on amq2 (ant consumer -Durl=tcp://0.0.0.0:61617)
7) Terminal 6: start up a producer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq3. The messages should be forwarded to either amq1. The consumer connected to amq1 should have received the 2000 messages and shut down.

8) Terminal 1: shut down amq1 (ctrl-c)

Note the logging that shows the failover taking place successfully. Let's test it to see if the demand forwarding bridge actually got started.

9) Terminal 6: start up a producer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq3. The consumer connected to amq2 receives the 2000 messages and shuts down.

10) Terminal 1: start up amq1 (./bin/activemq)

11) Terminal 2: shut down amq2 (ctrl-c)

Again, the failover took place successfully. Let's continue just a bit further to see if it will continue to failover if I bring up amq1 again.

12) Terminal 4: start up a consumer on amq1 (ant consumer -Durl=tcp://0.0.0.0:61616)

13) Terminal 6: start up a producer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq3. The consumer connected to amq1 receives the 2000 messages and shuts down.

This proves that the failover transport is supported in a network connector and it does work correctly with a uni-directional network connector. In addition to a uni-directional network connector, I also tested a bi-directional network connector. This only requires a slight change to the configuration of the network connector in amq3. In the amq3 XML configuration file, in the network connector element, add a duplex=true attribute. Below is the network connector element for amq3 with the change:


<networkConnector name="amq3-nc"
uri="static:(failover:(tcp://0.0.0.0:61616,tcp://0.0.0.0:61617))"
dynamicOnly="true"
networkTTL="3"
duplex="true" />


With this minor change in configuration, the network connector is now bi-directional. I.e., communication between amq3 and whichever broker it connects to is two-way instead of just one-way. This means that messages can be sent in either direction, not just in one direction originating from amq3.

Below are the steps I used to test this specific change:

1) Open five terminal windows as defined below:
1a) Terminal 1 = cd into the amq1 dir
1b) Terminal 2 = cd into the amq2 dir
1c) Terminal 3 = cd into the amq3 dir
1d) Terminal 4 = cd into the amq1/example dir
1e) Terminal 5 = cd into the amq1/example dir

2) Terminal 1: start up amq1 (./bin/activemq)
3) Terminal 2: start up amq2 (./bin/activemq)
4) Terminal 3: start up amq3 (./bin/activemq)

You should see that amq3 makes a network connection to either amq1 or amq2. For the rest of these steps, let's assume that amq3 connected to amq1.

5) Terminal 4: start up a consumer on amq1 (ant consumer -Durl=tcp://0.0.0.0:61616)
6) Terminal 5: start up a producer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq3. The messages should be forwarded to amq1. The consumer connected to amq1 should receive the 2000 messages and shut down.

Let's test the duplex capability of the network connector in amq3 now. To do this we'll send messages to amq1 and consume those messages from amq3.

7) Terminal 4: start up a consumer on amq3 (ant consumer -Durl=tcp://0.0.0.0:61618)
8) Terminal 5: start up a producer on amq1 (ant producer -Durl=tcp://0.0.0.0:61616)

You should see 2000 messages sent to amq1. The messages should be forwarded to amq3. The consumer connected to amq3 should receive the 2000 messages and shut down. This proves that the duplex feature is working. Now let's cause a failover/reconnect to take place and run through this same set of steps with amq3 and amq2.

9) Terminal 1: shut down amq1 (ctrl-c)

Notice the logging that shows the failover taking place successfully so that amq3 connects to amq2 now.

10) Terminal 4: start up a consumer on amq2 (ant consumer -Durl=tcp://0.0.0.0:61617)
11) Terminal 5: start up a producer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq3. The messages should be forwarded to amq2. The consumer connected to amq2 should receive the 2000 messages and shut down.

Now let's test the duplex feature in the network connector.

12) Terminal 4: start up a producer on amq2 (ant consumer -Durl=tcp://0.0.0.0:61617)
13) Terminal 5: start up a consumer on amq3 (ant producer -Durl=tcp://0.0.0.0:61618)

You should see 2000 messages sent to amq2. The messages should be forwarded to amq3. The consumer connected to amq3 should receive the 2000 messages and shut down.

This proves that the duplex feature of the network connector works after a failover/reconnect to amq2.

This is a great addition to ActiveMQ that really improves the usability of a network of brokers. I already have some very large clients using this feature successfully, some of which are using a network of over 2000 brokers.

Hopefully these steps are clear enough to follow for your own use. If you need any clarifications, please contact me.

12 November 2009

Grrr - What Happened to Java 1.5 on Snow Leopard?!



Just this week I updated to Mac OS X Snow Leopard (10.6.2) only to discover that it removes Java 1.5 entirely - WTF?! I have no idea why Java 1.5 was removed but it was a bad decision. How is it logical that Java 1.3 is reinstalled but Java 1.5 is removed? Anyway, after some searching and a bit of trial and error on my system, here are steps that I had to take to fix this situation:


  1. Download Java for Mac OS X 10.5 Update 4

  2. IMPORTANT!!! Make sure to run the following two commands BEFORE proceeding:

    1. sudo rm /System/Library/Frameworks/JavaVM.framework/Versions/1.5

    2. sudo rm /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0


    This removes the old Java 1.5 sym links to prevent the Java 1.6 directory from being overwritten (very bad)

  3. Using the Finder, locate the JavaForMacOSX10.5Update4.dmg file and double-click on it to mount the disk image

  4. Create a sym link to a missing utility:

    1. sudo ln -s /usr/bin/update_dyld_shared_cache /usr/bin/update_prebinding


    This command fixes a missing sym link in Snow Leopard that is needed by the Pacifist in the next step

  5. Using the Pacifist, open the JavaForMacOSX10.5Update4.pkg file from the mounted disk image

  6. In the Pacifist, navigate to /System/Library/Frameworks/JavaVM.framework/Versions and follow these steps:

    1. Right-click on 1.5 and select 'Install to Default Location'

    2. Right-click on 1.5.0 and select 'Install to Default Location'




After going through these steps, Java 1.5 should now be installed on the system. You can see this by listing the following:


$ ls -l /System/Library/Frameworks/JavaVM.framework/Versions/1.5
lrwxr-xr-x 1 bsnyder staff 5 Nov 12 13:47 /System/Library/Frameworks/JavaVM.framework/Versions/1.5 -> 1.5.0
$ ls -1 /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/
Classes
Commands
Headers
Home
Libraries
Resources




NOTE: Through my travels, I've found that many people have no idea that you could switch between different versions of Java. To make the task of switching extremely easy, grab my friend David's setjdk script and use it to handle this task from the command line. It's a bash script that even supports tab completion. So once the script is set up in your environment, switching between versions of Java is as easy as running the following command to see the available Java versions:

$ setjdk <tab>

Then just choose the version you'd like to use, e.g.:

$ setjdk 1.5
Setting this terminal's JDK to 1.5 ... java version "1.5.0_19"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02-304)
Java HotSpot(TM) Client VM (build 1.5.0_19-137, mixed mode, sharing)


UPDATE: I had to use the steps above to reinstall the documentation for Java 1.5 as well. I found it in the Java for Mac OS X 10.5 Update 4 Developer Documentation. This placed the appledocs.jar, the doc.jar and the src.jar files into the /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home directory. Then I just expanded the doc.jar file to be able to browse the API docs for Java 1.5 via a browser.

20 June 2009

SpringSource University in Denver



On Tuesday morning I had just returned from a run at about 9:35am and was checking my messages before I showered and began working for the day. As I was reading various messages, dripping sweat and sipping on some water, I was pinged by my friend and colleague Filip Hanik on IRC. He told me that he was sitting in a classroom in Denver for the SpringSource University Core Spring Training. Here's how the conversation went:


Filip: hey
Bruce: hey, what's up?
Filip: sitting at Core Spring in Denver, but there is no trainer
Bruce: whoops
Bruce: sorry to hear that, is there anything I can do to help?
Filip: yeah, you can come down here and teach this course :)
Bruce: are you serious?
Filip: yes
Bruce: headed for the shower now, see you about 11


By 11am I was in Denver and teaching the first day of the course.

Lucky for me, Chris Beams, a stellar consultant/engineer/trainer at SpringSource, was en route to deliver the rest of the course. Chris arrived later Tuesday evening and taught the course for the rest of the week through Friday and I was lucky enough to sit in. Chris went the extra mile to arrive early and stay late for the rest of the week and even wound up finishing on time on Friday afternoon, a testament to his talents, for sure.

Using Maven? You Should Be Using m2eclipse



If you use Maven for your Java projects, then you should take a look at m2eclipse plugin for Eclipse. After all, there's now a whole book on m2eclipse instead of just a single chapter!

11 June 2009

Enterprise Integration Patterns in Practice at JavaOne 2009



Last week I was in San Francisco to speak at JavaOne 2009 to deliver a talk named Enterprise Integration Patterns in Practice where I co-presented with Andreas Egloff from Sun.

We talked about implementing EIP patterns using Apache Camel and Sun's Fuji. I used ActiveMQ in my demo which was rather short but worked well (yay! the demo gods smiled on me last week :-) ) and Andreas showed off Sun's web-based EIP editor. Although this editor looks very cool and Andreas showed how to edit scripting code in each component on the diagram, it left me wondering how it interfaces with Java code. After our demos, my laptop would not flip back to the presentation at the end (which presented an abrupt ending) but still we received a lot of complements throughout the rest of the week.

Beyond the talk, I got to see tons of friends who I don't see that often throughout the year including Debbie, Guillaume, Hiram and Rob, as well as many new SpringSource colleagues (since the acquisition of Hyperic). I also met many, many new folks including Colin and Jamie. I was also lucky enough to eat Thai food twice (love the yellow curry) and sushi once during the week - mmmmmmmmmmm.

If you would like a copy of the presentation, just drop me an email and I'll send it your way (bruce DOT snyder AT gmail DOT com). I'm not sure how much sense the slides will make without the delivery, but it seems like folks are always requesting my presentations after the fact.

12 May 2009

JeOS, Oracle XE and VMWare on MacOS X

Last week I installed JeOS (Just Enough Operating System) on VMWare Fusion. JeOS is a slimmed-down version of Ubuntu specifically made for installation in a virtual machine so that it has a much smaller footprint than the normal Ubuntu distribution. Anyway, I did this so that I'd have a VM with a smaller footprint that I can use on my MacBook Pro for development and testing.

After setting up JeOS, I installed Oracle XE on it. Oracle XE is a slimmed down version of Oracle that is free for development and distribution. I used these instructions for installing Oracle XE on Ubuntu but there were some catches, namely that by default the web application for Oracle XE was bound to the localhost, i.e., I couldn't see the web app from Firefox in MacOS X. This was easily remedied via this comment on a blog post about this very topic:

Set up the environment on JeOS to use sqlplus:


export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server
export PATH=${PATH}:${ORACLE_HOME}/bin


Now use sqlplus to alter a setting that disables local-only access to Apex:


$ sqlplus system@xe
Enter password: SYSTEM_password

SQL> EXEC DBMS_XDB.SETLISTENERLOCALACCESS(FALSE);


Using the Apex webapp (http://:8080/apex) I was able to a user for myself. After this, I could access Oracle XE from MacOS X.

This setup has been working very well for me until today. For no apparent reason, the networking in JeOS went flaky and I could not access the outside world at all. I switched between NAT and host-only networking in VMWare multiple times, rebooting the VM each time - no dice. I upgraded to the latest VMWare Fusion - no dice. I could SSH into the VM from MacOS but that was it. No commands from JeOS would reach the outside world. So on a whim, I reran the vmware-install.pl script again and after that the networking seems to work again. I still have no idea why this happened which is a bit troubling. I don't want to just blindly keep running the install script again with no idea why it's fixing this networking issue.

Update

I guess the networking issue is common enough that Fusion provides a script to restart everything underneath of Fusion, e.g.:


/Library/Application\ Support/VMware\ Fusion/boot.sh --restart


Also, there's a good document available for Understanding Networking in VMware Fusion if you're interested. It's brief and to the point.