Pages

Friday, August 28, 2009

Well, I din expect i will have a topic to write so soon after join for MS course. But, thanks to Dr. Li I had a chance to work with java again!!! I love java! So, here goes a tutorial which explains how to use javacvs lib.

I will explain important snippets here, will provide the complete source at the end of the tutorial.

Step 1: Create the CVS Root object defining the meta data such as username, host etc To do this, we have to read the CVS/Root file.

LogListenerIt extends BasicListener, and overrides the messageSent with an empty definition. It also overrides the fileInfoGenerated method, where the actual processing of each file is done. That is log each of file. The snippet is below.

//Handle control to Super class. super.fileInfoGenerated(fileinfoevent);

logToExcel is a simple method that writes the log to the file. The datastructure LogInformation has data related to the file being processed, all the revisions of the file, head revision, the message associated with each revision and lot of more details. I use only the ones I mentioned.The complete code including the LogListener is below. Ignore the logic that is being done at logToExcel method, that some thing related to my work. But, the goal is to know that fileInfoEvent has LogInformation, that can be used to see data. You can write different listeners, builders in the similar way.

/* * Create a client that can execute the cvs log command, and register the listener to be invoked for CVS outputs. */ Client client = new Client(connection, new StandardAdminHandler()); client.setLocalPath(cvsPath); client.getEventManager().addCVSListener(new LogListener());

/* * The log command that does the job of getting the log for every file in the * directory. * The log builder is responsible to invoke fileinfo events on the loglistner. */ LogCommand command = new LogCommand(); command.setRecursive(true); Builder builder = new LogBuilder(client.getEventManager(), command); command.setBuilder(builder);

Monday, July 27, 2009

I quit my company Tavant, and have decided to pursue my higher education, MS in Compuer Science at University of Alabama, Huntsville. So, this blog may not be updated at a great frequency. When I work on some thing cool, will definitely update the Tech details here.

You can follow my other blog if you are interested at http://sprouting-beans.blogspot.com/

Thursday, June 18, 2009

I have worked on couple of push technologies namely blazeDS and Lightstreamer and always wondered how does it work. I am just concerned about streaming in Flex and not anything else. You can think I am little inclined towards flex, but the truth is Flex rocks, and after I entered the world of Flex, other UI technologies - I don't care anymore! Anyways, this logic of streaming data to Flex from a backend came to me just by chance! It took 2 days effort to get the below crappy thing going! I read a blog on the Internet that used URLStream to load image data in Flex progressively. Now, if you can load image progressively, then I should be able to do the same with any data. So, here I present Flex - XSocket combination to subscribe/load data from backend for a watchlist. There are a few things, that I did not resolve, as I did not have time to dedicate for it. I will explain those at the end.

JAVA

Lets start from the java backend. I use XScoket Xlightweb library to create a NIO server that can take any symbol as subscription and pushes randomly generated quotes to the client.

Lets write a request handler. In XLightweb, the request handler implements the IHttpRequestHandler.

public class ERequestHandler implements IHttpRequestHandler{ //Timer that generates the quotes at the specified interval private final Timer timer = new Timer("timer", true); public static Map<String, SymbolSender> state = new HashMap<String, SymbolSender>();

Lets implement the onRequest method. The same handler will be used to receive commands, like add symbol, remove symbol and also to receive data request, we use the URI to differentiate between the two request.

Every ipAddress is given its own handler. So, obtain the handler ie SymbolSender and delegate the action to it. The SymbolSender is a private class that is given little below. Before any client sends an action, it has to connect to this server.

The action has been delegated, so close the channel with a 200 Response header as in the below snippet.

IHttpResponseHeader respHdr = new HttpResponseHeader(200, "text/plain"); BodyDataSink outChannel = exchange.send(respHdr); outChannel.close(); }else if(req.getRequestURI().endsWith("/link2")){//The initial Data link, that is used to send data back. String command = req.getParameter("action"); Initialize a new SymbolSender if none exist for the given ipAddress and start the handler using the timer. Here, it is scheduled at a fixed rate of every 500ms.

The stream listener for the data is given below. Very simple logic, just read the multi byte which is already written in form of utf-8 by the server and append it to the data string. Now, this stream listener is called by Flex when there is some specific amount of data is present. And, it does not make sense for me to parse the data here to create objects at that rate. Instead as you will see next code, the timer will invoke the parse method every 60ms and then parse the data in the data string to an appropriate VO.

Now, the timepass listener is here. Comment the alerts if you do not want, or use trace instead.

private function listener(event:Event):void{ Alert.show("Here at other side of the world :("); }

Now, here is the parse method for the data I stream from the server. Actually, I do not like to call it streaming, as it is just a normal read and write to streams. Heck! that's why we call it streaming. He he..

Find out if the data is written partially. That means if its not ending with "|". If that is the case then keep the last piece of string in the data string for appending with the next data arrived from the backend. This will be parsed after the data has arrived and 60ms time lag.

Parse all the data other than "STAY ALIVE" pings. I am not sure if you can stop sending STAY ALIVE or delay its frequency. And, I am not doing that R & D either. My purpose remains just of stream data to Flex. Only the positive use case :) The rest of the for loop is just parsing data, creating or updating the StreamerVO object.

Next lets write the 4 actions supported namely add, delete, start and stop. The url's are in sync with the actions supported by respective links given in the backend code. The get parameters are constructed manually as shown below.

And, that's all folks. The backed is writing the data to the channel, and front end flex is reading it and displaying. Well, there are few bugs in this code I presented.

The data does not start to flow to the front end for initial 20 sec or so. Try to connect and subscribe say 3-4 symbols IBM, GOOG, MSFT, CSCO soon. I guess the problem is with the Operating system buffer because even when I said flush manually in the XSocket code (Application buffer) no effect could be seen or the problem could exist in Flex also, I am not sure. As I said earlier, my goal is not to solve these problems but to rather get streaming up, which gets up after a initial delay which is fine with me right now.

I had to use two links so that I could add symbols to be subscribed with out re-connecting to get the data that is being pushed. I could not figure out a way to do it using a single link. You guys can give a shot and let me know. I leave this here for now, as I am in no mood of writing a complete infrastructure for push when several exists.

Below is a small video of the Flex UI using the code above showing streaming data.

Wednesday, June 17, 2009

Writing an MBean in java is very simple. But, when you want the return type to a table of data(say, some metrics) then simple Mbeans are not of much use. One way to get table of data as a return type is using a Open Mbeans. This tutorial gives a simple example of how to write an Open Mbean that returns Table of data.

All the variables can be made private and initialized either in a static block or in the constructor. The CompositeType just defined that each row will contain the columns "symbol", "localTimeStamp", "latestResponse". Now lets define the table type it self.

OpenMBeanParameterInfo[] params = new OpenMBeanParameterInfoSupport[0]; operations[0] = new OpenMBeanOperationInfoSupport( "resetAndGetQuoteSnapshot", "Reset and get the latest available data for the Quotes", params, quoteSnapshotType, MBeanOperationInfo.INFO); //Build the info

This is all is needed to build an Open MBean that supports a Tabular data as output. You can use jconsole and view the output. Obviously, you have to deploy the MBean in mbean server. I use spring to do that job. The spring XML is given below.

Saturday, April 04, 2009

I know i am posting this second part pretty late, but can't help, have loads of work at hand. So, here goes the next part of the Lightstreamer tutorial which explains the Flex part. Remember, the flex client swc is available only if you download the evaluation version of the Lightstreamer. This is not packaged with the Moderato version.

This tutorial will explain how to use the native AS3 Flex client lib to subscribe data from the Adapter that I present in the previous post. Strictly speaking, this post will just provide a beginning to LS, we have developed a complete framework on the Flex side which is being used by around 9 modules transparently with out the knowledge of existence of LS. Well, I will not and I am not supposed to provide any more details about it.

Change the port at which the LS server runs. Edit the $LS_HOME/conf/lightstreamer_conf.xml. Change <port>8080</port> to <port>8787</port> in the HTTP server configuration present. Restart the server.

Step 1:

Use the LSClient to connect to the LS server which has the adapter deployed.

var connectionInfo:ConnectionInfo = new ConnectionInfo(); connectionInfo.setServer("localhost");// Place where the server is running connectionInfo.setAdapter("MyJavaAdapter"); // This is the name of the adapter we configured in the adapters.xml connectionInfo.setControlPort("8787");// The port we configured above. connectionInfo.setPort("8787");// The port we configured above. connectionInfo.setControlProtocol("http");// The protocol to use http or https. For quotes data, http is preferred. connectionInfo.setProtocol("http");

var connectionPolicy:ConnectionPolicy = new ConnectionPolicy(); //These are more configurations, as needed by the application. connectionPolicy.setTimeoutForStalled(2); connectionPolicy.setTimeoutForReconnect(15); connectionPolicy.setRetryTimeout(5);

Step 2:Subscribe to the data. You can use Visual or NonVisual table. We use only the NonVisual table. ok, I am sure you did not understand what is a table. In a simple term, a table contains rows and columns. In LS term, the columns are the items you subscribe for. For e.g. LastPrice. The rows are the items you subscribe. For E.g. Quote data for IBM. So, each row is a unique subscription for the items present as columns. Non-Visual table is not a component that will be displayed on the UI. You handle the display yourself. I feel this is more flexible and powerful than Visual Table.

Ok, here I will explain some terms. The last param in the constructor arg of NonVisualTable is the management mode. There are 4 types, namely, RAW, DISTINCT, MERGE and COMMAND. You can look at documentation for detailed explanation of all these.In simple terms, if your data is constantly changing like quotes, and LS sends only those data that have changed. The data that is not changed is not sent and its state is maintained as previous. i.e the last unchanged value. This type of management of data is called MERGE mode. If you need the data as being generated, i.e each of the data should be pushed to the front end, like news data, then you use the DISTINCT mode. RAW mode is similar to Distinct. Each and every data is sent to the client. I am not so sure, but the only difference the document says is, in case of DISTINCT not all the events be sent to the client. COMMAND is more complicated and has better performance, please read the documentation for it. I have used MERGE and DISTINCT mode till now successfully.

The DataAdapter is the one you configured in adapters.xml, in our case "MYJAVAADAPTER". You can request a snapshot if the data-adapter supports one.

The requested max frequency limits the updates I get from the LS server. I do not need more than 1 update per item per second. So, its 1 in my case. Make sure you use this with MERGE mode only.

handleUpdate gets the updates from the LS. It is given below. The code is self explanatory.

This is all you need to get going on the Flex side with LS. We started from here and you wont believe where we have reached. The LS forums is really really helpful. Please post any queries there, I have always got sensible and useful replies. Try going through the lightstreamer_conf.xml, there are a lot of things you can configure there. Any developer with some experience will understand it. If you need more help, just go to the forums.

This will end the Lightstreamer tutorials from my side. Try this technology, am sure you will love it...

Monday, February 09, 2009

I think there is shortage of tutorials on web related to Lightstreamer, which is one awesome technology. Initially you may become over whelmed at the terminologies and trying to understand each and every part of it. I use lightstreamer as a part of architecture of a trading platform. I have used it pretty extensively within limits for the application. Hence, I would like to give this tutorial for people who wants to use this push software in there app's. I will cover the concepts with some amount of examples that will make ME understand Lightstreamer. :)

The basic requirement to understand this tutorial is, you need to know the concept of push technology first. I have used BlazeDS Messaging initially, so, may be if you are a Flex developer, first check out BlazeDS messaging. Then understand the concept of Push technology. Then, come to this tutorial to make more sense out of it.

In Lightstreamer, there are two parts for the push (I will call it 'streaming') to work. A backend and a front end. The backend is explained in java and front end is explained in Flex.

This 1st part of Tutorial is about Java Backend. I will explain the Flex Client side UI in the next part of the Tutorial.

Installation:

Download the zip from http://www.lightstreamer.com/. Extract it in a directory which I will call $LS_HOME, and read the GETTING_STARTED.txt in $LS_HOME. Its neatly and clearly written. There should not be a problem in understanding it. I would suggest you download a trial version than using the Moderato, the free one. And, Also buy it if you find it very useful :) Anyway's, its valid for 60 days, which is more than enough for you to evaluate it and understand its capability.

The Backend:

Structure: The first thing is, where do you put the code that has been written and how do you put it. There are two approaches. First one is, compile your java code, get the .class files, and put it under $LS_HOME/adapters/myadapter/classes. Second one is, create a jar out of your class files, and put it in $LS_HOME/adapters/myadapter/lib.

Create the two folders myadapter and classes/lib as needed by your approach.

Once the structure is ready, copy the adapters.xml from the samples given in $LS_HOME/adapters/Demo.

So, what is Adapter? Simplest term, this is what is responsible for pushing your data to your UI.

Lets write an adapter. Also, there are three more terminologies which we need to know.

2. itemName: This is any String that can be used to uniquely identify the type of data that has been requested by the client to the server. The server, i.e this adapter should understand the data that client from UI is requesting to be pushed. In the example below, it is as simple as Symbol IBM, that the server pushes data to UI. 3. ItemEventListener: This is the interface defined by Lightstreamer(LS) that is instantiated by LS, when the LS server is started. This has a method 'update', that is used to push data to the UI. 4. Snapshot: If the data is not available to be pushed, but a snapshot of it is available then the isSnapShotAvailable method can return true, and the itemEvenListener can send true in the last arguments for snapshots. The UI can request for a snapshot if it needs it. In trading sample app that I give below, a snapshot is not of much use, so, is defined as false.

And, this is it, we are good to go for an example.

Its very simple, create a Class MyJavaAdapter and implement com.lightstreamer.interfaces.data.DataProvider. That is it.. :)

Below is a simple adapter, that can be used generically with any Class that extends DataGenerator abstract class and is capable of generating data from any source, either database or from another server that pumps data over a network. You can define your own implementation.

In the code sample below, the MyJavaAdapter delegates its work to any class that extends DataGenerator. DataGenerator, does the job of setting and working with Listener given by the LS. Any class, that extends the DataGenerator is hidden from the intricacies of the LS. The push method in the DataGenerator does the actual job of pushing data to the UI. The data is sent in form of a key=value pair. The extended class, has to worry just about its business logic, i.e, where to get data from and push it to the server by converting it to a map. The sample below is a very very simple design, and can be highly modified/replaced to make it more generic and better, I hope you get the point.

The sample QuotesStubber that extends DataGenerator is given below. It uses timer to generate data at regular interval and push it to the front end. The push is happening in the run method if you observe.

The Adapter is below, this is responsible for interactions with the DataGenerator given above. The methods defined here is called by LS server at appropriate times. The reference for the datagenerator in the adapter below can be either injected or instantiated as given in comment.

You will use the 'name' attribute defined here from the Flex UI when you start requesting data to be pushed to it.The Flex part of it will be provided in my next post/tutorial as a continuation to this one.