Apr 23, 2013

Understanding the MapReduce concept

Step 1 is mapping: Take one input, process into many outputs. For example, take some wooden blocks and split them into smaller blocks. Process them to have key/value pairs.

Step 2 is reducing: Process those many outputs, for example process each block by coloring them in different colors and then generate one output by combining the split blocks with similar color into one block by gluing them together.

Why do we need to map it and then reduce it? Because if the data sets are so huge, you can process them in parallel by different machines by splitting up the tasks to make it more efficient. The data is split across server nodes. This is the MapReduce way of processing large data. Not every piece of data can be easily split and then combined.

Example: Input = "big brown fox jumped over a brown fence". The requirement is to count the number of occurrences of the word "count".

Note: In Windows under file explore, the hidden files might not be displayed by default. ".svn" is a hidden file, and to turn it on you need to click on Tools --> Folder Options and then select "View" and ensure that "Show hidden files and folders" is selected.
Q. How will you list the ip address of your machine?A.Use the ipconfig command

to list all the accounts. This is a plural resource returning a collections of accounts. The URI contains nounsrepresenting the resources in a hierarchical structure. For example, if you want a to get a particular transaction value under an account you can do

Where 123 is the account number and 567 is the transaction number or id. This is a singular resource returning a single transaction.
What if you want to list a collection of transactions that are greater than a particular date?

The verbs are definedvia the HTTP methods GET, POST, PUT, and DELETE. The above examples are basically GET requests returning accounts or transactions. If you want to create a new transaction request, you do a POST with the following URL.

The actual transaction data will be sent in the body of the request as JSON data.The above URI will be used with a PUT http method to modify an existing transaction record.

Finally, you can also control which method gets executed with the help of HTTP headers or host names in the URL.Let's see some Spring MVC examples in a controller class as to how it maps a request URI, headers, etc to execute the relevant method on the server side.

Don't ever use GET to alter state. Use GET for as much as possible. Favor POST over PUT when in doubt.

Don't perform an operation that is not idempotent with PUT.

Do use DELETE in preference to POST to remove resources.

Don't clutter your URL with verbs or stuff that should be in a header or body. Move stuff out of the URI that should be in an HTTP header or a body. Whenever it looks like you need a new verb in the URL, think about turning that verb into a noun instead. For example, turn 'activate' into 'activation', and 'validate' into 'validation'.

The http://www.wiztools.org/ has some handy open source Java tools like RESTClient, Regular Expression Tester, etc. RESTClient tool is GUI based and a good alternative to the Unix command line based tool CURL and the Firefox poster plugin. SoapUI is another GUI based client for both RESTful and SOAP based web services. This blog posts shows how easy it is to get started with RESTClient tool from WizTools. The example below shows a POST request as the GET requests are easier to test. The diagrams below illustrates posting of JSON data. The RESTful web service operations GET, POST, PUT, and DELETE correlates with the database CRUD operations Read, Create, Update, and Delete respectively.

Step 1: From WizTools click on the link to RESTClient.

Step 2: Download the JAR file "restclient-cli-3.1-jar-with-dependencies.jar". Create a short cut to the jar file. Double clicking on the short cut will open the RESTClient GUI as shown below. You can type in the URL and select "POST" as the HTTP method.

Take note of the HTTP headers passed with the request.

Step 3: You need to select the "Content-Type" that you want to post. Select "application/json" as the content-type.

The GET requests are very straight forward requiring only the URL.

The SoapUI tool requires a WADL file for the RESTFul web services. Here is an example of SOAP Web Service using the SoapUI tool.

Unix is very powerful, and you can achieve a lot with less amount of code. Most production systems run in a Unix environment, and it is very useful to know your commands to solve technical issues. Some handy Unix commands looping, recursive search, control characters, and crontab.Q. How will you go about concatenating the contents in a number of text/sql files to a single file?
The account_sqls.patch file contains the following entries to a number of SQL files to be executed sequentially.

1, 4p // print lines 1 to 4 from account_sqls.patch for sql in $(sed -n '1,4p' account_sqls.patch); // for each line read do cat $sql >> bulk_data_changes.sql; // append the contents of each file to a different file accounts_combined.sql.Note: This is handy for combining contents from a number of different files.

Q. How will you recursively search for presence of a particular text under multiple folders unix? For example, search for text "CashEnum" within a java project?

A. Use the find and grep commands. The find will look for the files and the grep will filter out the files that has the text "CashEnum".

find myproject -type f -exec grep "CashEnum" {} /dev/null \;

-type f --> find files.

the addition of /dev/null is so that grep will consistently print the file name when it finds a match.

The above command will search all file extensions, including .jar. You could restrict it to look for only java files as shown below

Q.How will you go about executing multiple maven commands on unix? For example, build two separate projects one after another, for example -- project1 and project2. The project2 to should only build if project1 successfully builds.

A. Use the && control operator to combine two commands so that the second is run only if the first command returns a zero exit status. In other words, if the first command runs successfully, the second command runs. If the first command fails, the second command does not run at all. For example:

Similarly, the || control operator separates two commands and runs the second command only if the first command returns a non-zero exit status, that is if it fails. In other words, if the first command is successful, the second command does not run. This operator is often used when testing for whether a given directory exists and, if not, it creates one.

cd project1/src/main || mkdir -p project1/src/main

the -p command allows you to create any relevant parent directories as well. The && and || control characters can be used together.

Q. How will you check using unix commands that a given class file exists within a jar file?A. Let us assume that there are a number of jar files sitting under the myjars/lib folder. The following script "search-class-file.sh" sits under myjars.

Apr 10, 2013

Maven - setting up profiles and tokenizing properties file

Maven is a popular build tools and often you have a requirement to tokenize the name/value pairs in a properties file so that right values can be substituted depending on for which environment (local, dev, test, uat, prod) you are building the package for.

The values within "@" are tokens that needs to be replaced at build time. The properties files reside under src/main/resources folder as per the Maven structure.Here is a sample DEV resource myapp.properties that sits under /env/dev folder.

Q. Can you give an example where you used a profile?A. Firstly, to tokenize properties as demonstrated in the beginning of the post. Secondly, in local development you set up a datasource for your tomcat server to use as shown below.

Apr 9, 2013

JavaScript Tutorial -- coding and debugging with FireBug

Step 1: Download and install Mozilla Firefox (latest available version). Also install the add on FireBug. Two other handy add on plugins for the FireBug are YSlow and Cookies.

The small "Split Screen Screen" button in FireBug as shown above will allow you to have a split screen. You can use the right hand side screen to type in you JavaScript code and execute it. The left hand side screen can be used for displaying the console.log statements and errors.

FireBug is a very handy tool to inspect your DOM elements and CSS styles. You can also add break points to your JavaScript for debugging. The FireBug can be turned on and off with the F12 button.Google chrome has similar development tools and you can bring it on with the F12 key.

Step 3: Here is the sample function that uses both jQuery and JavaScript as shown below. This function compares two objects or values and checks if both objects or values are same. It returns either true or false as the return value. This function is not fully tested and used for illustration purpose only. Feel free to try different scenarios and fix any issues that arise.

Step 4: Invoke the The above code can be typed in the FireBug console as shown below for debugging and execution.

Now, you can follow the above steps to write a JavaScript function and debug the function using FireBug. Also try using the FireBug to inspect HTML and CSS code by clicking on the respective tabs. The Script tab is also used for placing break points and debugging JavaScript.

Q. How will you go about writing an SQL query for the following scenario?

Valuation table with the following columns portfolioid, accountid, balance, inactiveflag, valuationdttm, and typecd. The portfolio table has columns portfolioid, and portfoliocd. The account table has columns accountid and accountcd.

Write an SQL query to extract out the accountcd and the corresponding balance for a given portfoliocd and valuationdttm. Please note that there will be multiple balance records for each account, and your query must only extract out a single balance record per account based on the rule 'extract the record with minimum value for typecd'.

A. As you can see in the sample answer below, inner joins are used to join with the relevant tables. A sub query is used to calculate the min(typecd) to extract the record with minimum value for typecd.

Q. If you need to map actual values retrieved from the database to some other value and then sort by these translated values as well, how will you go about accomplishing this in your SQL code?

For example, StatusCd is the column in the Portfolio table, and it can have the values of New, and Processed. But the SQL query should return a status of 'Excluded' if the ExcludedFlag column is set yes, and 'Sent' if the SentDateTime is not null. iIf none of the above conditions are met, then return the StatusCd as in the database. The sorting needs to be carried out in the order of 'New', 'Processed', 'Sent', and then 'Excluded'.

A. This can be achieved with a switch/case statement. The syntax of switch/case statement can vary among databases. Here is a sample SQL based on Sybase database server.

SELECT PortfolioCd, SentDateTime, ExcludedFlag, StatusCd as ActualStatusCd,
case when p.ExcludedFlag = 'Y' then 'Excluded'
else case when p.SentDateTime is null then p.StatusCd
else 'Sent'
end
end as EvaluatedStatusCd
FROM Portfolio p WHERE valuationdttm > '09 Jan 2013' and InActiveFlag = 'N'
ORDER BY case when p.ExcludedFlag = 'Y' then '4'
else case when p.SentDateTime is not null then '3'
else case when p.StatusCd = 'New' then '1'
when p.StatusCd = 'Processed' then '2'
end
end
end,
PortfolioCd

Q. How would you retrieve a date time column converted to string and formatted as dd/mm/yy hh:mm:ssA. You can use specif functions provided by your database server. These functions are specific to the database server you are using, hence your code cannot be ported to other database servers. Here is an example in Sybase.

Apr 4, 2013

Top 3 Core Java Third Party librararies that you use frequently in your Java projects

Q. Can you list a few third party libraries and useful methods that you frequently use in you core Java and enterprise Java development?A. Firstly, the Apache Commons library has a number of utility methods to simplify your development with a number of proven APIs with utility methods.

Firstly, the toString( ), equals( .. ), and hashCode( ... ) methods are very frequently used in your domain or value object classes. Here are a few handy Apache commons library methods.

Package org.springframework.util has a number of handy utility classes like Assert, ClassUtils, CollectionUtils, FileCopyUtils, FileSystemUtils, ObjectUtils, StringUtils, TYpeUtils (Java 5 Generics support) and ReflectionUtils to name a few. The Assert utility class that assists in validating arguments. Useful for identifying programmer errors early and clearly at run time. For example,

Assert.notNull(clazz, "The class must not be null");
Assert.isTrue(i > 0, "The value must be greater than zero");

The google Guava library provides a number of precondition checking utilities with the Preconditions class.

@Controller
...
public void order(ProductInfo data) {
//required in the controller to handle errors and send a sensible error back to the client
Preconditions.checkArgument(StringUtils.hasText(data.getName()), "Empty name parameter."); //validation
productService.order(data);
}

The Service layer needs to apply validation to promote defensive programming. The design by contract states that your methods should check for pre and post conditions and fail fast if the contract is not met.

If you have a requirement to work a lot with the Java collections, and performance and memory usage are of utmost importance, then look at the following libraries to work with the collections.

The Trove library provides high speed regular and primitive collections for Java.

PCJ (Primitive Collections for Java)

Guava (from google)

In a nut shell, the Apache commons from Apache, Spring utils from Spring, and Guava libraries from Google should make you more productive without having to reinvent the wheel. If you have heavy usage of data structures, then Trove becomes very handy. So, before you roll out your own utility methods, check if they are already provided by these libraries.

Q. What libraries or frameworks do you use to marshaling and unmarshaling Javas POJOs to CSV, XML, EXCEL, and PDF?A. There are a number of libraries available.

BeanIO is an open source Java framework for reading and writing Java beans or plain old java objects (POJO's) from a flat file or stream. BeanIO is ideally suited for batch processing, and currently supports XML, CSV, delimited and fixed length file formats. It is also handy for writing automated tests using Apache Camel where test instructions can be written on a flat file and read via BeanIO. It can also be used to produce CSV and XML download files.

Apr 3, 2013

CSS Interview Questions and answers

Modern applications are web driven using HTML, CSS, and JavaScript. HTML is the building block of Web pages. CSS is the language used to make those Web pages pretty. JavaScript is used for dynamically manipulating the DOM elements and their look and feel. Understanding the basics of HTML and CSS will help you build better Web pages. CSS (Cascading Style Sheets) are vary useful and it it really worth understanding its key concepts. This blog explains the key concepts via questions and answers style.

Q. For the HTML snippet below, can you answers the questions listed below.

#myAppDetails : refres to an element with id myAppDetails, which is the top "DIV" element.form : refers to a child form element under the element with id "myAppDetails".button : refers to a child button element under the form element in the hierarchy..submit or .refresh : refers to an element with a style class, which has the value of either "submit" or "refresh" in the hierarchy under the button element.

Finally the the "display:none" will hide the element selected.

Q2. In the above example, if you want to select more specifically the form element that has the style class submit, and a child element button with class submit or refresh, how will you write the CSS selector to display the selected element?

The above CSS is more specific than the previous one. The previous one selects any form, but this one selects only the form element with a style class of submit.

#myAppDetails : select any element with id "myAppDetails".form.submit : select an element that has a style class "submit".button : a button child element under a form with class "submit".submit or .refresh : a child element with a class name either "submit" or "refresh" under a button element.

You can define multiple selectors with a comma.

Q3. What if the button element must immediately follow the form element?A3. You can use the symbol ">" to denote immediate. This is not used frequently. Here is the sample.

Q4. Why does your CSS rules don't apply to some elements?A4. You can minimize the time it takes for hunting bugs by understanding how the browsers interpret your code and using developer tools like firebug for Firefox and Google chrome developer tool.

Every CSS selector has its place in the specificity hierarchy. If two selectors apply to the same element as demonstrated in the above example, the one with the higher specificity wins. For example,

"#myAppDetails form.submit button .submit" is more specific than "#myAppDetails form button .submit" because the first one looks for specific form with class value of submit, whereas the second one applies to any form.

There are four distinct categories which determine the specificity level of a given selector: inline styles, IDs, classes+attributes and elements. ID selectors have a higher specificity than attribute selectors. You should always try to use IDs to increase the specificity. A class selector beats any number of element selectors. Here is an example:

<div id="myDivId" style="font-color:blue" class="aCSSClass" >

In CSS you will have

.aCSSClass {
font-weight:12;
}

/** CSS comment: attribute css selector **/ [title] {
color:blue
}

You can calculate the specificity yourself by allocating some weights, but the better approach is to use developer tools like Firebug or Chrome developer tool to calculate it for you and display the effective CSS.

Q5: When will you use a float?A5: A float is used for laying out elements with CSS in a Web page.

Q6: Where can you apply a float?A6: You can't float every element on a Web page. To get technical, you can only float block level elements. These are the elements that take up a block of space on the page, like ,

,

, etc. The other elements known as inline elements like , , , etc can't be floated.

Q7: What are some of the basics you need to keep in mind regarding floats?A7:

When you float an element it becomes a block box. This box can then be shifted to the left or right on the current line. The markup options are float: left, float: right or float: none.

You should always set a width on floated items, except if applied directly to an image, which has an implicit width. If no width is set, the results can become unpredictable.

The elements can only be floated to the left or right, it is not possible to make an element float in the centre. When we float an element it is shifted to the right or to the left until it reaches the edge of the containing block.

Elements following a floated element will wrap around the floated element. If you do not want this to occur, you can apply the “clear” property to these following elements. The four options are clear: left, clear: right, clear: both or clear: none.

If the container element is the HTML , the floated div will sit on the left margin of the page. If the container element is itself contained by something else, the floated div will sit on the left margin of the container.

Q8: What tips would you give to someone who is using CSS in their projects?A8: