Sunday, August 12, 2012

Writing Unit Tests to test JMS Queue listener code with ActiveMQ

During my GSoC project for OpenNMS i did some work with JMS and wanted to write test classes to make sure that my code was working well. In this article i will try to explain how you can use ActiveMQ to write test cases for your JMS code. This is very handy when your code has a part that listens to a JMS queue.

Apache ActiveMQ 

Apache ActiveMQ is a extremely popular and very powerful open source messaging and Integration Patterns server. You can check it out here. The part we will be using to run our test classes is the embedded broker that is provided by ActiveMQ. This allows you to create a temporary broker for test purposes to create a JMS queue in our case. If you want to learn more about the embedded broker check out this article. The easiet way to create a embedded broker is through the following code line. This will automatically create a embedded broker.
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");

Code to be tested

Before we take a look at the test code lets take a look at the code that we are trying to test. here the purpose of code is to listen to a given JMS Queue and do some processing on the received data. to allow smooth operation of the listener code we will need to create 3 classes.

The Server class this is the main class that will listen to the Queue. below is the code snippet that will start the JMS Queue connection.
public void startServer(){
        try{
            ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");

            // create a queue connection
            QueueConnection queueConn = (QueueConnection) connectionFactory.createConnection();

            // create a queue session
            QueueSession queueSession = queueConn.createQueueSession(false,
                                                                     Session.DUPS_OK_ACKNOWLEDGE);

            Destination destination = queueSession.createQueue("TESTQUEUE");


            // create a queue receiver
            MessageConsumer consumer = queueSession.createConsumer(destination);

            // set an asynchronous message listener
            MeasurementListner measurementListner = new MeasurementListner();
            consumer.setMessageListener(measurementListner);

            // set an asynchronous exception listener on the connection
            MeasurementExceptionListener measurementExceptionListener = new MeasurementExceptionListener();
            queueConn.setExceptionListener(measurementExceptionListener);
            queueConn.start();

        }catch(Exception e){
            logger.error("JMS Exception",e.getMessage());
        }
To make sure that the server is running continually you can use a wait method as given below and call it just after calling the startServer() method. If you only want to run the server for a limited time you can use a Thread.sleep(timeinmillies).
 synchronized void waitForever()
    {   
        while (true) {
            try {
                wait();
            } catch (InterruptedException ex) { }


        }
    }
The other 2 classes that you will need is a class that implements the MessageListener interface and ExceptionListener interface the two classes MeasurementListner and MeasurementExceptionListener are those two classes, the onMessage(Message message) method is called when a message is received from the JMS Queue. So we want to test if the code actually gets messages from the defined queue that why we need the ActiveMQ embedded broker.

Writing the Test code
 public class ListnerTest {

    private String jmsQueue = "TESTQUEUE";
    private String url = "vm://localhost?broker.persistent=false";
    @Test
    public void testMeasurment(){

        try {
            .........
            String string = new String("Hello");
           
            sendMessage(string);

            Server server = new Server();
            server.main(null);
            ........
            
        } catch (MessageConversionException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 

    }
   

    private void sendMessage(Object obj){
        try{
            
            SimpleMessageConverter smConverter = new SimpleMessageConverter();
            ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
            Message message;

            
            QueueConnection queueConn = (QueueConnection) connectionFactory.createConnection();
            queueConn.start();

            QueueSession queueSession = queueConn.createQueueSession(false,
                                                                     Session.DUPS_OK_ACKNOWLEDGE);

            Destination destination = queueSession.createQueue(jmsQueue);

            MessageProducer queueSender = queueSession.createProducer(destination);
            queueSender.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
            
            message = smConverter.toMessage(obj, queueSession);
            queueSender.send(message);
            
        }catch (Exception e) {
            e.printStackTrace();
        }
    }

}

The test code simple creates a String and writes into the JMS Queue created using the embedded broker of ActiveMQ. i am calling the main method of the server since the startServer() call and wait call is made in the main method. If you got everything running you should be able to get a nice unit test running for your project. I hope this was helpful to anyone trying to do this. please leave a comment if you need any clarification i can help out if possible.


Friday, August 10, 2012

Setting up a ZooKeeper Quorum on Amazon EC2 with Exhibitor

Hi in this post i will try to explain step by step how to set up a ZooKeeper Quorum in Amazon EC2 with the help of Exhibitor. This was done as a part of my final year project so i thought i should share this because it might be helpful to others. I assume that you know about ZooKeeper otherwise you want be reading this article right. anyway if you want to learn about ZooKeeper you can check out the ZooKeeper home page here.

Exhibitor

So i will first introduce you to Exhibitor its actually a supervisory system that is built using java technology to monitor and supervise ZooKeeper. It provides some cool features that will help you manage your ZooKeeper instances. You can download Exhibitor here. And if you want to learn more about Exhibitor take a look at there wiki page here. Exhibitor allows you to configure ZooKeeper instances through a very clean web interface. the config panel will look like this.


As you can see you can define the data dir and the ZooKeeper installation dir. lets talk about the server list after we get the servers set up in Amazon EC2.

Setting up Servers in Amazon EC2

First you will have to create an account at Amazon to get Amazon web services such as EC2. You will need an credit card to create an account but all the work needed to get ZooKeeper running can be done under the free tier so it want cost you anything. So when you got your account set up you can find your way into the EC2 console. In the console you can create new instances, i want go into details on how to set up servers, Amazon has pretty neat set of documentation so you can refer them here. So using the console create 3 Ubuntu servers preferably Ubuntu 12.04 Servers.

I assume that you used the quicklaunch security group or the default security group when creating the servers. Its fine even if you created a custom group. You can change the setting in your security group from the Security Groups option in the navigation panel. So go over there and change the rules to free the ports that you will be using, keep the source as 0.0.0.0/0. you will need to free the ssh port 22 and other ports that you will be using in Zookeeper and 8080 for Exhibitor. if you are feeling lazy you can just use the All TCP  rule in the drop down list.

You can connect to the server through ssh. how to do this is explained step by step here.

Installing ZooKeeper and Exhibitor

Now that you can access  the servers lets install ZooKeeper and Exhibitor. create a folder in the home dir and download ZooKeeper and Exhibitor and extract them. Create a dir to be used as the data dir for ZooKeeper in my case i used "/home/ubuntu/zookeeper/data/" you have to repeat this for all 3 servers. And since we are using ZooKeeper in the replicated mode you will need to create the myid files and assign 1,2 and 3 for the three servers.

Installing Java and setting classpath variables 

You will have to install Java jre and jdk before you can run anything on the server. it would be better to install sun-java6 take a look at my previous article if you run into trouble installing sun-java6 on the server.

Now you have to add the following lines into the .bashrc file in your home dir.

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.26/
export PATH=$PATH:/usr/lib/jvm/java-6-sun-1.6.0.26/bin:$JAVA_HOME/bin
export CLASSPATH=/usr/lib/jvm/java-6-sun-1.6.0.26/lib

And at last you need to comment the following line that can be found on the top of the .bashrc file or the settings will not be available non-interactive calls.

[ -z "$PS1" ] && return

Lets get things running
Now go into Exhibitor and run the Exhibitor jar file if you want to keep it running even after you close your ssh connection use the command "nohup" like "nohup java -jar ....". Now you should be able to access Exhibitor through your browser using the public DNS of the server it should be somthing like "ec2-xx-xxx-xx-xxx.us-west-2.compute.amazonaws.com" . So the URL for Exhibitor will be something like the following. and you do this for all three servers and get the interface for each server.

http://ec2-xx-xxx-xx-xx.us-west-2.compute.amazonaws.com:8080/exhibitor/v1/ui/index.html

In the config tab enter the ZooKeeper installation dir, in my case it is "/home/ubuntu/zookeeper/zookeeper-3.4.2/" and you can also set the data dir.  You can change the additional config information if you like i am using the following settings.

initLimit=10
syncLimit=5
tickTime=2000

Set the client port to the port you want i am using 2185 for this one. you can change them to what you like and they can be different in each 3 Exhibitor instance. The connect port and the election port need to be same in all three config's so i will be using 2900 and 3900 in all 3 instances.

The server list

Now the server list is a comma separated string of the ip of the servers. lets say the following are the public and private ip's of the three servers. these can be found from the Amazon console.

1. public ip - ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-1x-xxx-x-xxx


2. public ip - ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-2x-xxx-x-xxx

3. public ip - ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-3x-xxx-x-xxx



So the server list of the first server will be ( the one with myid set to 1 in the data dir ).

1: ip-1x-xxx-x-xxx,2:ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,3:ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com

second server

1:ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,2:ip-2x-xxx-x-xxx,3:ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com

third server

1:ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,2:ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,3:ip-3x-xxx-x-xxx

After you get the config's set properly click commit on each exhibitor interface. after a few minutes  you should be able to see the three servers in the control panel tab. So now you should have your ZooKeeper Quorum up and running under Exhibitor.


 If you find this post interesting or run into and problems when following the post please leave a comment. I may be able to help you because i ran into a ton of errors while i was setting this up for the first time.


pulasthi

Amazon Deals