Sunday, August 12, 2012

Writing Unit Tests to test JMS Queue listener code with ActiveMQ

During my GSoC project for OpenNMS i did some work with JMS and wanted to write test classes to make sure that my code was working well. In this article i will try to explain how you can use ActiveMQ to write test cases for your JMS code. This is very handy when your code has a part that listens to a JMS queue.

Apache ActiveMQ 

Apache ActiveMQ is a extremely popular and very powerful open source messaging and Integration Patterns server. You can check it out here. The part we will be using to run our test classes is the embedded broker that is provided by ActiveMQ. This allows you to create a temporary broker for test purposes to create a JMS queue in our case. If you want to learn more about the embedded broker check out this article. The easiet way to create a embedded broker is through the following code line. This will automatically create a embedded broker.
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");

Code to be tested

Before we take a look at the test code lets take a look at the code that we are trying to test. here the purpose of code is to listen to a given JMS Queue and do some processing on the received data. to allow smooth operation of the listener code we will need to create 3 classes.

The Server class this is the main class that will listen to the Queue. below is the code snippet that will start the JMS Queue connection.
public void startServer(){
        try{
            ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");

            // create a queue connection
            QueueConnection queueConn = (QueueConnection) connectionFactory.createConnection();

            // create a queue session
            QueueSession queueSession = queueConn.createQueueSession(false,
                                                                     Session.DUPS_OK_ACKNOWLEDGE);

            Destination destination = queueSession.createQueue("TESTQUEUE");


            // create a queue receiver
            MessageConsumer consumer = queueSession.createConsumer(destination);

            // set an asynchronous message listener
            MeasurementListner measurementListner = new MeasurementListner();
            consumer.setMessageListener(measurementListner);

            // set an asynchronous exception listener on the connection
            MeasurementExceptionListener measurementExceptionListener = new MeasurementExceptionListener();
            queueConn.setExceptionListener(measurementExceptionListener);
            queueConn.start();

        }catch(Exception e){
            logger.error("JMS Exception",e.getMessage());
        }
To make sure that the server is running continually you can use a wait method as given below and call it just after calling the startServer() method. If you only want to run the server for a limited time you can use a Thread.sleep(timeinmillies).
 synchronized void waitForever()
    {   
        while (true) {
            try {
                wait();
            } catch (InterruptedException ex) { }


        }
    }
The other 2 classes that you will need is a class that implements the MessageListener interface and ExceptionListener interface the two classes MeasurementListner and MeasurementExceptionListener are those two classes, the onMessage(Message message) method is called when a message is received from the JMS Queue. So we want to test if the code actually gets messages from the defined queue that why we need the ActiveMQ embedded broker.

Writing the Test code
 public class ListnerTest {

    private String jmsQueue = "TESTQUEUE";
    private String url = "vm://localhost?broker.persistent=false";
    @Test
    public void testMeasurment(){

        try {
            .........
            String string = new String("Hello");
           
            sendMessage(string);

            Server server = new Server();
            server.main(null);
            ........
            
        } catch (MessageConversionException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 

    }
   

    private void sendMessage(Object obj){
        try{
            
            SimpleMessageConverter smConverter = new SimpleMessageConverter();
            ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
            Message message;

            
            QueueConnection queueConn = (QueueConnection) connectionFactory.createConnection();
            queueConn.start();

            QueueSession queueSession = queueConn.createQueueSession(false,
                                                                     Session.DUPS_OK_ACKNOWLEDGE);

            Destination destination = queueSession.createQueue(jmsQueue);

            MessageProducer queueSender = queueSession.createProducer(destination);
            queueSender.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
            
            message = smConverter.toMessage(obj, queueSession);
            queueSender.send(message);
            
        }catch (Exception e) {
            e.printStackTrace();
        }
    }

}

The test code simple creates a String and writes into the JMS Queue created using the embedded broker of ActiveMQ. i am calling the main method of the server since the startServer() call and wait call is made in the main method. If you got everything running you should be able to get a nice unit test running for your project. I hope this was helpful to anyone trying to do this. please leave a comment if you need any clarification i can help out if possible.


Friday, August 10, 2012

Setting up a ZooKeeper Quorum on Amazon EC2 with Exhibitor

Hi in this post i will try to explain step by step how to set up a ZooKeeper Quorum in Amazon EC2 with the help of Exhibitor. This was done as a part of my final year project so i thought i should share this because it might be helpful to others. I assume that you know about ZooKeeper otherwise you want be reading this article right. anyway if you want to learn about ZooKeeper you can check out the ZooKeeper home page here.

Exhibitor

So i will first introduce you to Exhibitor its actually a supervisory system that is built using java technology to monitor and supervise ZooKeeper. It provides some cool features that will help you manage your ZooKeeper instances. You can download Exhibitor here. And if you want to learn more about Exhibitor take a look at there wiki page here. Exhibitor allows you to configure ZooKeeper instances through a very clean web interface. the config panel will look like this.


As you can see you can define the data dir and the ZooKeeper installation dir. lets talk about the server list after we get the servers set up in Amazon EC2.

Setting up Servers in Amazon EC2

First you will have to create an account at Amazon to get Amazon web services such as EC2. You will need an credit card to create an account but all the work needed to get ZooKeeper running can be done under the free tier so it want cost you anything. So when you got your account set up you can find your way into the EC2 console. In the console you can create new instances, i want go into details on how to set up servers, Amazon has pretty neat set of documentation so you can refer them here. So using the console create 3 Ubuntu servers preferably Ubuntu 12.04 Servers.

I assume that you used the quicklaunch security group or the default security group when creating the servers. Its fine even if you created a custom group. You can change the setting in your security group from the Security Groups option in the navigation panel. So go over there and change the rules to free the ports that you will be using, keep the source as 0.0.0.0/0. you will need to free the ssh port 22 and other ports that you will be using in Zookeeper and 8080 for Exhibitor. if you are feeling lazy you can just use the All TCP  rule in the drop down list.

You can connect to the server through ssh. how to do this is explained step by step here.

Installing ZooKeeper and Exhibitor

Now that you can access  the servers lets install ZooKeeper and Exhibitor. create a folder in the home dir and download ZooKeeper and Exhibitor and extract them. Create a dir to be used as the data dir for ZooKeeper in my case i used "/home/ubuntu/zookeeper/data/" you have to repeat this for all 3 servers. And since we are using ZooKeeper in the replicated mode you will need to create the myid files and assign 1,2 and 3 for the three servers.

Installing Java and setting classpath variables 

You will have to install Java jre and jdk before you can run anything on the server. it would be better to install sun-java6 take a look at my previous article if you run into trouble installing sun-java6 on the server.

Now you have to add the following lines into the .bashrc file in your home dir.

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.26/
export PATH=$PATH:/usr/lib/jvm/java-6-sun-1.6.0.26/bin:$JAVA_HOME/bin
export CLASSPATH=/usr/lib/jvm/java-6-sun-1.6.0.26/lib

And at last you need to comment the following line that can be found on the top of the .bashrc file or the settings will not be available non-interactive calls.

[ -z "$PS1" ] && return

Lets get things running
Now go into Exhibitor and run the Exhibitor jar file if you want to keep it running even after you close your ssh connection use the command "nohup" like "nohup java -jar ....". Now you should be able to access Exhibitor through your browser using the public DNS of the server it should be somthing like "ec2-xx-xxx-xx-xxx.us-west-2.compute.amazonaws.com" . So the URL for Exhibitor will be something like the following. and you do this for all three servers and get the interface for each server.

http://ec2-xx-xxx-xx-xx.us-west-2.compute.amazonaws.com:8080/exhibitor/v1/ui/index.html

In the config tab enter the ZooKeeper installation dir, in my case it is "/home/ubuntu/zookeeper/zookeeper-3.4.2/" and you can also set the data dir.  You can change the additional config information if you like i am using the following settings.

initLimit=10
syncLimit=5
tickTime=2000

Set the client port to the port you want i am using 2185 for this one. you can change them to what you like and they can be different in each 3 Exhibitor instance. The connect port and the election port need to be same in all three config's so i will be using 2900 and 3900 in all 3 instances.

The server list

Now the server list is a comma separated string of the ip of the servers. lets say the following are the public and private ip's of the three servers. these can be found from the Amazon console.

1. public ip - ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-1x-xxx-x-xxx


2. public ip - ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-2x-xxx-x-xxx

3. public ip - ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com
    private ip - ip-3x-xxx-x-xxx



So the server list of the first server will be ( the one with myid set to 1 in the data dir ).

1: ip-1x-xxx-x-xxx,2:ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,3:ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com

second server

1:ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,2:ip-2x-xxx-x-xxx,3:ec2-3xx-xxx-xx-xx.us-west-2.compute.amazonaws.com

third server

1:ec2-1xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,2:ec2-2xx-xxx-xx-xx.us-west-2.compute.amazonaws.com,3:ip-3x-xxx-x-xxx

After you get the config's set properly click commit on each exhibitor interface. after a few minutes  you should be able to see the three servers in the control panel tab. So now you should have your ZooKeeper Quorum up and running under Exhibitor.


 If you find this post interesting or run into and problems when following the post please leave a comment. I may be able to help you because i ran into a ton of errors while i was setting this up for the first time.


Thursday, May 24, 2012

Installing sun-java-6 with apt-get in Linux


Recently i ran into some trouble trying to install sun-java-6 into a Linux server because newer Ubuntu versions do not provide sun-java-6 packages. After some digging i was able to find a solution. You can get it working by adding new sources into apt-get sources.
 There are two ways to get it done easily

1. by running add-apt-repository command
 sudo add-apt-repository "deb http://ppa.launchpad.net/ferramroberto/java/ubuntu oneiric main"

If you do not have add-apt-repository already installed you can install it by doing
 sudo apt-get install python-software-properties

2. by directly changing the source list file
 sudo vim /etc/apt/sources.list

Then add the line into the file.
 deb http://ppa.launchpad.net/ferramroberto/java/ubuntu oneiric main

After you have added this line into the sources from any one of the above methods. perform
 sudo apt-get update

Now you should be able to install sun-java-6 without any problem with apt-get install. 

Wednesday, February 1, 2012

Debugging Zookeeper with eclipse

This will be a very small post but i thought i should post it because the time i wasted trying to do this simple thing. what i wanted to do was to run the Zookeeper JUnit test one by one so i could debug them to understand how the system works by looking at how the test cases run. I use Eclipse as my IDE so i wanted to do this using Eclipse.

In Eclipse if there are JUnit test classes in the source code you can run a single test class by opening the class ( the .java ) right click on it and go to Debug As --> JUnit Test and simple run the tests in that test class.

I'm also a novice in this area so i am sure i might have made a mistake or two but i couldn't get Debug As ---> JUnit Test to appear when i checkout the project using subeclipse. So this is the way that worked for me.

Checkout Zookeeper

Just checkout Zookeeper from svn using the following command.
svn checkout http://svn.apache.org/repos/asf/zookeeper/trunk/

Build Eclipse project files

Then go to the trunk  folder ( using the terminal ) and execute the ant command "ant eclipse" this will generate the files needed for an eclipse project.

Import the project into Eclipse

After you run the ant command you can import it into. go to File --> Import in the pop box that appers select "Existing Projects into Workspace" under "Generel"


Click next and in the next window set the root folder of the project that you checked out and import the project. Now you should be able to run JUnit tests comfortably. 

Sunday, January 29, 2012

Creating a Group Chat bot with Google App Engine

Hi in this post i will try to explain how to develop a chat bot for gmail actually this will work with any chat application that supports XMPP. Here i will explain how to develop a group chat application that can be used to chat with a group of people. I know you must be wondering why i would go through the trouble to make and app when gmail already supports group chats. Well the problem was that since our final year project was a 4 member group project we had a lot of online discussions but whenever someone would get disconnected they will loose the messages sent while they were offline and someone will have add that person back into the group. so one of my friends suggested me to write a app to solve this because i had written a small chat app before. Lets get started.

What is Google App Engine

I will assume that you are a bit familiar with Google App Engine. Well if you are not please take a look here to get an general understanding you only need the basic knowledge to follow this post.

Create a new Application in App Engine

Before we start you should create a new application in your App Engine account. Lets say the application name of my app is "gmailgroupchat" so the identifier of my app will be "gmailgroupchat.appspot.com". Now since we have an application id for our app lets see how to develop the app.

Setting up Google App Engine plugin in Eclipse.

Before you can start developing an application for the Google App Engine we have to set the working environment. I use eclipse as my IDE you can find the instruction to install the plugin here. There is also a plugin  available for Netbeans you can find the instructions for Netbeans here.

Creating a new project.

I assume that you have installed the App Engine plugin for Eclipse by now. to create a new project use File --> New --> Web Application Project. In the window that pops up set the name of your project this does not need to be same as the app id also untick the Google Web Toolkit box. Eclipse will generate the folder structure with some sample code.

Now to allow the App Engine to identify what this app is we have to set our app id in the "appengine-web.xml" that is located in war/WEB-INF

  <application>gmailgroupchat</application>
  <version>1</version>

Here the version indicates the version of your application App Engine gives you the ability to keep a number of versions of the same app.

NOTE: You will get and error of you try to deploy an application with a version number that already exists.

Now lets look at the Servlet code that will do all the work. I will explain what each part does step by step first lets look at the whole code.
public class GmailGroupChatServlet
    extends HttpServlet {
  
  private static final Logger LOG = 
      Logger.getLogger(GmailGroupChatServlet.class.getName());
  
  @Override
  public void doPost(HttpServletRequest req,
      HttpServletResponse resp) throws IOException {
    
    // Parse incoming message
    XMPPService xmpp = XMPPServiceFactory.getXMPPService();
    Message msg = xmpp.parseMessage(req);
    String body = msg.getBody();

     JID jid = msg.getFromJid();
     JID jid1 = new JID("email1@gmail.com");
     JID jid2 = new JID("email2@gmail.com");
     JID jid3 = new JID("email3@gmail.com");
     JID jid4 = new JID("email4@gmail.com");
    
   
    
    String response = "";
    if(jid.getId().split("/")[0].equals("email4@gmail.com")){
     response = "*Email4:* " + body;
      Message msg1 = new MessageBuilder()
         .withRecipientJids(jid1)
         .withBody(response)
         .build();
         xmpp.sendMessage(msg1);
      Message msg2 = new MessageBuilder()
         .withRecipientJids(jid2)
         .withBody(response)
         .build();
         xmpp.sendMessage(msg2);
      Message msg3 = new MessageBuilder()
         .withRecipientJids(jid3)
         .withBody(response)
         .build();
         xmpp.sendMessage(msg3);
     
     
    }else if(jid.getId().split("/")[0].equals("email3@gmail.com")){
     response = "*Email13:* " + body;
     Message msg1 = new MessageBuilder()
       .withRecipientJids(jid1)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg1);
    Message msg2 = new MessageBuilder()
       .withRecipientJids(jid2)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg2);
    Message msg3 = new MessageBuilder()
       .withRecipientJids(jid4)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg3);
    }else if(jid.getId().split("/")[0].equals("email2@gmail.com")){
     response = "*Email2:* " + body;
     Message msg1 = new MessageBuilder()
       .withRecipientJids(jid1)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg1);
    Message msg2 = new MessageBuilder()
       .withRecipientJids(jid3)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg2);
    Message msg3 = new MessageBuilder()
       .withRecipientJids(jid4)
       .withBody(response)
       .build();
       xmpp.sendMessage(msg3);
    }else if(jid.getId().split("/")[0].equals("email1@gmail.com")){
     response = "*Email1:* " + body;
     Message msg1 = new MessageBuilder()
     .withRecipientJids(jid2)
     .withBody(response)
     .build();
     xmpp.sendMessage(msg1);
  Message msg2 = new MessageBuilder()
     .withRecipientJids(jid3)
     .withBody(response)
     .build();
     xmpp.sendMessage(msg2);
  Message msg3 = new MessageBuilder()
     .withRecipientJids(jid4)
     .withBody(response)
     .build();
     xmpp.sendMessage(msg3);
 
    }


  }
}

The following lines will parse the input message and save the string. Since we are developing a group chat what we have to do is simply forward the message to all the other members.
    XMPPService xmpp = XMPPServiceFactory.getXMPPService();
    Message msg = xmpp.parseMessage(req);
    String body = msg.getBody();

In the Follwing lines we define the id's of the users of the group chat. since this is made to be used with the 4 members of our group the id's are hard-coded. the created JID's will later be used to send the incoming messages to the other members.In the first line we capture the id of the message sender.

  
     JID jid = msg.getFromJid();
     JID jid1 = new JID("email1@gmail.com");
     JID jid2 = new JID("email2@gmail.com");
     JID jid3 = new JID("email3@gmail.com");
     JID jid4 = new JID("email4@gmail.com");

Then what we need to do is to make sure the sender does not get his own message back. The if conditions check the id of the message sender and makes sure that the sender does not get his own message. The String split is used because JID will contain additional information other than the email. The message is sent by this code.Here "withRecipientJids" sets the JID of the recipient of the message.
  
    Message msg1 = new MessageBuilder()
     .withRecipientJids(jid2)
     .withBody(response)
     .build();
     xmpp.sendMessage(msg1);

NOTE: The Servlet uses the doPost method because XMPP expects a post call.

Now to add the magic code that allows your Servlet to receive chat messages sent by users. Your "web.xml" should contain this code fragment that binds the Servlet to the magic URL pattern
  
   <servlet>
  <servlet-name>GmailGroupChatServlet</servlet-name>
  <servlet-class>com.appengine.capp.GmailGroupChatServlet</servlet-class>
 </servlet>
 <servlet-mapping>
  <servlet-name>GmailGroupChatServlet</servlet-name>
  <url-pattern>/_ah/xmpp/message/chat/</url-pattern>
 </servlet-mapping>

"/_ah/xmpp/message/chat/" is the magic URL pattern.Before we can deploy the application there is one more insertion we need to do in to the "appengine-web.xml" to enable XMPP for the java app. The following code fragment should be included.
  
  <inbound-services>
    <service>xmpp_message</service>
  </inbound-services>

Now you can deploy the application in to the App Engine to do this right click on the project select Google --> Deploy to App Engine. You will have to authenticate your self if you are doing it for the first time.

How to use the app.

In order to use this chat application each user has to send a invitation to the app requesting to chat in this example the chat id of the application will be "gmailgroupchat@appspot.com" because the identifier was 
"gmailgroupchat.appspot.com".

My first chat bot was a dictionary application that uses a Google API to reply with the meaning and pronunciation of the word entered to the chat bot . If someone is intersted you can use it by sending a chat request to "chatdictionary@appspot.com".

For further information on XMPP support in App Engine go here

pulasthi

Amazon Deals