Very handy quick reference guide and a road map where it gives a bird's eye view of the core concepts relating to Java/JEE and its related technologies. Why wait for years to learn the core concepts relating to Java/JEE, when you can pro-actively learn and apply them in months?

"Very well written! authors have good, clear style" I'm happy to say I know almost all of the content in the book already, but I'm still very happy I bought it. What really jumped out at me is how well written the explanations are. There are so many book out
there that get quite pompous and long-winded with their explanations of polymorphism, for instance. But the authors have summarize what polymorphism does in one short, clear sentence early on in the book. Most of the other explanations are similar, they really "cut through the crap" and provide the most direct answer possible. I recommend this book as a refresher for any Java professional--inevitably, we all get tied up in the peculiar...." --
Lulu.com reviews

"If you have read the The Pragmatic Programmer: From Journeyman to Master, you would enjoy this book too. Is similar about the comfortable pleasure to read this book." IT Book Zone Review

"If a person manages to give answers as given in book, he/she can easily bag a new job..." --
JavaRanch forum

"Core Java Career Essentials" are something you must know, and know it well to succeed as a Java professional. Even 40% to 60% of the so called experienced professionals fail to make an impression due to their lack of understanding in one or more of the areas covered in this book. Lack of good understanding in the core essentials can be a speed breaker to your career advancement.

If you have a java interview coming up in a few days or you want to impress your peers and superiors with your technical strengths during code review sessions, team meetings, and stand-ups, and concerned about how to make a good impression? You don't need to worry if you are familiar with the fundamentals. This is a Questions & Answers book with 250+ questions and answers relating to core Java with lots of code snippets (100+), examples, and diagrams. Most Java interviews and technical challenges you face at work are structured around the fundamentals and how well you communicate those fundamentals. So regularly brushing up on these fundamentals really pays off.

Specific technologies and frameworks come and go, but we believe the following are some of the key essentials that will not only help you become a good developer, but also will boost your career a few notches by impressing your superiors, peers, and prospective employers.

You might be excused for not knowing the flavor of the month framework, but you will not be excused for not knowing the core essentials.

Coding: You might be asked to demonstrate both recursive and iterative approaches to a given problem. You might be asked to critically review a given piece of code. You might be asked to write some sample code. 100+ code snippets highlighting the best practices and the pitfalls.

OO design:You will be asked to define basic OO concepts and principles, and come up with classes to model a simple problem.

Language fundamentals and best practices: You might be asked what parts of Java you don't like and why? You must know about the common pitfalls, anti-patterns, and how to avoid them.

Data structures: Most candidates know about arrays, sets, lists, and maps but fail to mention trees and graphs. You will be quizzed on real life examples of where to use a particular data structure.

Bits, bytes, and data types: You must know the logical operations like AND, OR, NOT, and XOR and where to use them in real life examples for things like setting or testing for a specific bit.

Right tools to use: Knowing the right tools will make you more productive without reinventing the wheel to get the job done more effectively and on time.

Hundreds of ready to use Java/JEE resume phrases. Use of Situation-Action-Result Statements. Results oriented and Key Areas driven. Soft skills and personal attributes are captured.

Shows you that a well written resume can invite you to interviews more often than many people more qualified than you.

Explains why technical skills alone are no guarantee of success and acquiring tons of certifications alone are not going to bring your next promotion or save you from being laid off.

Resume examples for freshers, career changers, intermediate, and experienced with eye catching phrases.

Ineffective resumes can result in prolonged job searches, fewer interviews, and very often, lower salary offers. Do you feel confident and good about your current Java/JEE resume and career progression? If you might no think so now, ask yourself the same question again after you have read this companion and have in your hands your completed, highly effective, customised, and well rounded resume.

Non-technical questions with hints

The open ended non-technical questions discussed below can reveal a lot about your passion and career aspirations.

Q. Why are you leaving your current job? What do you like and dislike about your current job?

A. Firstly, never be negative about your current or previous jobs. Also, don't bring salary into the discussion.

Likes:

Enjoyed the people I worked with.

Enjoyed solving technical and business problems.

Enjoyed identifying gaps in the business requirements and technical designs and getting them rectified collaboratively.

Enjoyed pro-actively identifying and fixing issues relating to non-functional requirements like performance, security, concurrency and memory leaks. These issues don't surface under normal scenarios and can be very challenging and satisfying.

Main reason for leaving is that I felt that I was not challenged enough at my current job. The company offered a great opportunity as someone with 2-3 year experience and appreciate the skills I acquired there, but now I am ready to face more challenges and take my career to the next level.

I am looking for more potential to grow in my next job.

I am also looking for an opportunity to get more involved with the hands-on design decisions in my next job.

Q. What was the biggest accomplishment and failure in your current job or career?

A. Reflect back on your career achievements and provide quantified answers.

Achievements:

Successfully completed a high profile Java/JEE based online trading system that handles 100+ concurrent users and integrates with 4 other systems.

Designed and developed a Java based non-blocking server that communicates with 240+ retail outlets, and integrates with 6 other internal systems.

Spearheaded a "Quick-Win" program that improved the site ranking from 24th to 9th in 3 months in terms of user experience, performance, and reliability.

Failures:

We all learn more from our mistakes or failures than from our successes or achievements. When you mention your mistakes or failures, make it a point to mention what you learned from them. You could say that these are classified more as mistakes than failures.

The team was using a cut down database in the development environment and the technical solution I provided worked well for low volume of data, but when was moved to production like data, it caused some performance issues. Learned a valuable lesson of validating the solution with more production like data set early on in the SDLC. I subsequently revised the solution within a week and got the the big thumbs up from the users.

Wrote some JUnit based integration tests that were bit fragile due to data fluctuations. The failing tests were causing the builds to fail. Took the initiative to fix this problem by performing the integration tests via more stable data sets that are populated via DBUnit during the set phase and the data are removed during the test tear down phase. Also, introduced other strategies for integration tests by using in memory databases like HSQL DB as opposed to actual database.

Q. How do you know what technologies, frameworks or tools are in demand?

A. Some technologies/frameworks don't even make it to the mainstream. So, do your research before learning.

1. Through online job portals and off line advertisements to understand what the prospective employers are looking for.
2. Through industry specific forums listed above to see what the experts are tipping.
3. Through some research via http://www.google.com/trends
4. Through personal experience and networking with fellow professionals.
5. Based on requirements searching for a right tool for the right job at

STEP 2: Read online articles, blogs, and books about a particular framework, technology, or tool to understand its high level architecture, core concepts, best practices, and potential pitfalls. Google is your best bet, but there are sites like

STEP 5: Find a way to apply or use a particular framework, technology, or tool in a commercial, self-taught, or open-source project.

STEP 6: Add this particular framework, technology, or tool to your resume's or CV's skills and experience section. Also, feel free to blog about your experience as you not only learn by blogging, but also increase your employ-ability by building an online persona.

Good programmers hate repetitive and monotonous tasks. They find the right tools and ways to automate these monotonous and repetitive tasks.

Good programmers don't reinvent the wheel. They will first look for the right tool, API, library, and framework to get the job done without requiring to write their own.

Q. What qualities do you look for in an effective programmer? How do you recognize a good programmer?

A. Passion, continuous learning, taking pride in their achievements, ability to look at the big picture and pay attention to details, right attitude, and ability to communicate their thoughts clearly.

Q.What do you understand by the term "hidden experience"?

A. Good programmers are mainly self-taught and they acquire lots of so called "hidden experience" through proactive and continuous learning and helping others solve their problems. If you just rely only on your experience alone, it can take a lot longer to learn the key areas and concepts of programming. Do you have real 4 year experience or 1 year repeated 4 times? It is imperative that you bring out your "hidden experience" in your CV and at job interviews.

Q. When you are are coding, what documentation do you have handy?

A.

1. The APIs for the relevant technologies used. For example Java API, Enterprise Java API, Spring API, jQuery API, JavaScript API, etc. Thes APIs can also be googled at will with the right search keys like "jQuery API", "Java 6 API" , etc.

2. The relevant reference manual and home web sites bookmarked for the relevant technologies. The home sites can be Googled with right keywords like "Spring framework home", "jQuery home", etc and the reference manual can be Googled with keywords like "Spring framework 2.0 reference",etc.

3. Often not knowing the key terms for a particular piece of technology is the major challenge for the beginners of a particular technology. This is where the "cheat sheets" come in very handy. You can find a number of handy cheat sheets from cheat sheets

Apr 24, 2012

Java interview questions and answers on asynchronous processing

The scenario based questions are very popular with the job interviewers, and some scenario based questions are related to decoupling and asynchronous (aka non-blocking) processing. The interviewer will give you a scenario like producing an aggregate report, designing a trading system, or building an application that requires heavy auditing, logging or custom metrics gathering. Feel free to ask the right questions as the interviewer will be more interested in how you think and approach a problem rather than you solve it instantly or not.

Q. Can you describe instances where you used asynchronous processing in your application? A. The asynchronous processing is very handy for decoupling systems and for boosting performance and scalability. Here are a few examples.

Example 1:

An online application with a requirement to produce time consuming reports or a business process (e.g. rebalancing accounts, aggregating hierachical information, etc) could benefit from making these long running operations asynchronous. Once the reports or the long running business process is completed, the outcome can be communicated to the user via emails or asynchronously refreshing the web page via techniques known as "server push" or "client pull". A typical example would be

a) A user makes a request for an aggregate report or a business process like rebalancing his/her portfolios. b) The user input can be saved to a database table for a separate process to periodically pick it up and process it asynchronously.c) The user could now continue to perform other functionalities of the website without being blocked.d) A separate process running on the same machine or different machine can periodically scan the table for any entries and produce the necessary reports or execute the relevant business process. This could be a scheduled job that runs once during off-peak or every 10 minutes. This depends on the business requirement.e) Once the report or the process is completed, notify the user via emails or making the report available online to be downloaded.

Example 2:

If you are working in an online trading application, you may want the functionality to queue trades and process them when the stock market opens. You also asynchronously receive the order statuses like partially-filled, rejected, fully filled, etc from the stock market. The message oriented middlewares provide features like guaranteed delivery with store-and-forward mechanism, no duplicates, and transaction management for enterprise level program-to-program communications by sending and receiving messages asynchronously (or synchronously). The diagram below gives a big picture.

Example 3:

You may have a requirement for stringent logging, auditing or performance metrics gathering. Processing these non-functional requirements asynchronously and non-intrusively can make your system perform and scale better. For example, you could send the log messages to a queue to be processed later asynchronously by a separate process running on the same machine or a separate machine. The performance metrics can be processed asynchronously as well. here is a working example with relevant code snippets.

For example, a trading application may have a number of synchronous and asynchronous moving parts and metrics needs to be recorded for various operations like placing a trade on to a queue, receiving asynchronous responses from the stock market, correlating order ids, linking similar order ids, etc. A custom metrics gathering solution can be accomplished by logging the relevant metrics to a database and then running relevant aggregate queries or writing to a file system and then running PERL based text searches to aggregate the results to a “csv” based file to be opened and analyzed in a spreadsheet with graphs. In my view, writing to a database provides a greater flexibility. For example, in Java, the following approach can be used.

Step 1: Use log4j JMS appender or a custom JMS appender to send log messages to a queue. This ensures that your application's performance is not adversely impacted by logging activities by decoupling it.

Step 2: Use this appender in your application via Aspect Oriented Programming (AOP – e.g Spring AOP, AspectJ, etc) or dynamic proxy classes to non-intrusively log relevant metrics to a queue. It is worth looking at Perf4j and context based logging with MDC (Mapped Diagnostic Contexts) or NDC (Nested Diagnostic Contexts) to log on a per thread basis to correlate or link relevant operations. Perf4J is a great framework for performance logging. It's non-intrusive and really fills the need for accurate performance logging. The Perf4j provides features like a command line tool to generate aggregated results and graphs from raw log files, ability to expose performance statistics as JMX attributes and to send notifications when statistics exceed specified thresholds, custom log4j appenders, and AOP aspects that allow non obtrusive statements when used with Spring AOP.

The Perf4J is for

System.currentTimeMillis( );

as Log4J is for

System.out.println(.....);

Step 3: A stand-alone listener application needs to be developed to dequeue the performance metrics messages from the queue and write to a database or a file system for further analysis and reporting purpose. This listener could be written in Java as a JMX service using JMS or via broker service like webMethods, TIBCO, etc.

Step 4: Finally, relevant SQL or regular expression based queries can be written to aggregate and report relevant metrics in a customized way.

An XML based library like Xtream can be used to serialize and deserialize the messages. Here is some pseudo code in Java.

Let's look at some sample code:

STEP-1: Define the custom appender in Log4j. In Log4J, appenders are responsible for handling the output and transport of logging data ('output destinations'). There are appenders for writing log data to files, for logging over the network or to write the logging output to the terminal, just to name a few.

Step 2: Define the interceptor that logs metrics using the custom JMS appender. The interceptor will be invoked on actual method invocation to log the time. The interceptor will forward the request to actual method call via invocation.proceed( ) method call. After invoking the actual method, the time duration for the method execution is calculated. The LOG.info method call will be using the previously defined JMS appender.

STEP 3: Write a stand-alone JMS listener to retrieve the XML event massages from the queue “jms/EventsQueue” and write to a database table or a file for further analysis and reporting. This can be done via a MDB (a Message Driven Bean in the EJB sepcification), a Spring based JMS listener, or a listener configured via a Message Oriented Broker like web Methods or TIBCO broker.

Apr 23, 2012

Java multi-threading interview questions and answers - continuation

Q. Why synchronization is important? A. Without synchronization, it is possible for one thread to modify a shared object while another thread is in the process of using or updating that object’s value. This often causes dirty data and leads to significant errors. The disadvantage of synchronization is that it can cause deadlocks when two threads are waiting on each other to do something. Also synchronized code has the overhead of acquiring lock, which can adversely affect the performance.

Q. What is a ThreadLocal class? A.ThreadLocal is a handy class for simplifying development of thread-safe concurrent programs by making the object stored in this class not sharable between threads. ThreadLocal class encapsulates non-thread-safe classes to be safely used in a multi-threaded environment and also allows you to create per-thread-singleton.

Q. What is a daemon thread? A. Daemon threads are sometimes called "service" or “background” threads. These are threads that normally run at a low priority and provide a basic service to a program when activity on a machine is reduced. An example of a daemon thread that is continuously running is the garbage collector thread. The JVM exits whenever all non-daemon threads have completed, which means that all daemon threads are automatically stopped. To make a thread as a daemon thread in Java

myThread.setDaemon(true);

The JVM always has a main thread as default. The main thread is always non-daemon. The user threads are created from the main thread, and by default they are non-daemon. If you want to make a user created thread to be daemon (i.e. stops when the main thread stops), use the setDaemon(true) as shown above.

Q. How can threads communicate with each other? How would you implement a producer (one thread) and a consumer (another thread) passing data (via stack)? A. The wait( ), notify (), and notifyAll( ) methods are used to provide an efficient way for threads to communicate with each other. This communication solves the ‘consumer-producer problem’. This problem occurs when the producer thread is completing work that the other thread (consumer thread) will use.

Example: If you imagine an application in which one thread (the producer) writes data to a file while a second thread (the consumer) reads data from the same file. In this example the concurrent threads share the same resource file. Because these threads share the common resource file they should be synchronized. Also these two threads should communicate with each other because the consumer thread, which reads the file, should wait until the producer thread, which writes data to the file and notifies the consumer thread that it has completed its writing operation.

Let’s look at a sample code where count is a shared resource. The consumer thread will wait inside the consume( ) method on the producer thread, until the producer thread increments the count inside the produce( ) method and subsequently notifies the consumer thread. Once it has been notified, the consumer thread waiting inside the consume( ) method will give up its waiting state and completes its method by consuming the count (i.e. decrementing the count).

Note: A method calls notify( )/notifyAll( ) as the last thing it does (besides return). Since the consume method was void, the notify( ) was the last statement. If it were to return some value, the notify( ) would have been placed just before the return statement.

Q. Why wait, notify, and notifyall methods are defined in the Object class, and not in the Thread class?A. Every Java Object has a monitor associated with it. The threads using that object can lock or unlock the monitor associated with the object.Wait and notify/notifyAll methods are responsible for acquiring and relinquishing the lock associated with the particular object. Calling wait causes the current thread to wait to acquire the lock of the Object, and calling notify/notifyAll relinquishes the lock and notify the threads waiting for that lock.

Q. What does join( ) method do? A. t.join( ) allows the current thread to wait indefinitely until thread “t” is finished. t.join (5000) allows the current thread to wait for thread “t” to finish but does not wait longer than 5 seconds.

For example, say you need to spawn multiple threads to do the work, and continue to the next step only after all of them have completed, you will need to tell the main thread to wait. this is done with thread.join() method.

Here is the RunnableTask. The task here is nothing but sleeping for 10 seconds as if some task is being performed. It also prints the thread name and timestamp as to when this task had started

The taskmanager manages the tasks by spawing multiple user threads from the main thread. The main thread is always created by default. The user threads 1-3 are run sequentially, i.e. thread-2 starts only after thread-1 completes, and so on. The user threads 4-6 start and executes concurrently.

Q. If 2 different threads hit 2 different synchronized methods in an object at the same time will they both continue? A. No. Only one thread can acquire the lock in a synchronized method of an object. Each object has a synchronization lock. No 2 synchronized methods within an object can run at the same time. One synchronized method should wait for the other synchronized method to release the lock. This is demonstrated here with method level lock. Same concept is applicable for block level locks as well.

Q. Explain threads blocking on I/O? A. Occasionally threads have to block on conditions other than object locks. I/O is the best example of this. Threads block on I/O (i.e. enters the waiting state) so that other threads may execute while the I/O operation is performed. When threads are blocked (say due to time consuming reads or writes) on an I/O call inside an object’s synchronized method and also if the other methods of the object are also synchronized then the object is essentially frozen while the thread is blocked.

Be sure to not synchronize code that makes blocking calls, or make sure that a non-synchronized method exists on an object with synchronized blocking code. Although this technique requires some care to ensure that the resulting code is still thread safe, it allows objects to be responsive to other threads when a thread holding its locks is blocked.

Q. If you have a circular reference of objects, but you no longer reference it from an execution thread, will this object be a potential candidate for garbage collection? A. Yes. Refer diagram below.

Q. Which of the following is true?

a) wait( ), notify( ) ,notifyall( ) are defined as final & can be called only from within a synchronized method
b) Among wait( ), notify( ), notifyall( ) the wait() method only throws IOException
c) wait( ),notify( ),notifyall( ) & sleep () are methods of object class

A. a and b. The c is wrong because the sleep method is a member of the Thread class.The other methods are members of the Object class.

Q. What are some of the threads related problems and what causes those problems?A. DeadLock, LiveLock, and Starvation.

Deadlock occurs when two or more threads are blocked forever, waiting for each other. This may occur when two threads, each having a lock on the same resource, attempt to acquire a lock on the other's resource. Each thread would wait indefinitely for the other resource to release the lock, unless one of the user processes is terminated. The thread deadlock can occur in conditions such as:

two threads calling Thread.join() on each other.

two threads use nested synchronized blocks to lock two objects and blocks lock the same objects in different order.

Starvation and livelock are much less common a problem than deadlock, and it occurs when all threads are blocked, or are otherwise unable to proceed due to unavailability of required resources, and the non-existence of any unblocked thread to make those resources available.

The thread livelock can occur in conditions such as:

all the threads in a program are stuck in infinite loops.

all the threads in a program execute Object.wait(0) on an object with zero parameter. The program is live-locked and cannot proceed until one or more threads call Object.notify( ) or Object.notifyAll() on the relevant objects. Because all the threads are blocked, neither call can be made.

Starvation describes a situation where a thread is unable to gain regular access to shared resources and is unable to make progress. This happens when shared resources are made unavailable for long periods by "greedy" threads. For example, suppose an object provides a synchronized method that often takes a long time to return. If one thread invokes this method frequently, other threads that also need frequent synchronized access to the same object will often be blocked. The thread starvation can occur in conditions such as:

one thread cannot access the CPU because one or more other threads are monopolizing the CPU.

setting thread priorities inappropriately. A lower-priority thread can be starved by higher-priority threads if the higher-priority threads do not yield control of the CPU from time to time.

Q. What happens if you call the run( ) method directly instead of via the start method?A. Calling run( ) method directly just executes the code synchronously (in the same thread), just like a normal method call. By calling the start( ) method, it starts the execution of the new thread and calls the run( ) method. The start( ) method returns immediately and the new thread normally continues until the run( ) method returns. So, don't make the mistake of calling the run( ) method directly.

Note: These Java interview questions and answers are extracted from my book "Java/J2EE Job Interview Companion".

If you liked the above Java multi-threading questions and answers, you will like the following link, which has a little more advanced coding questions on multi-threading.

Opportunities are limitless. As a freelance Java developer (aka Java contractor) since 2003, I have explored different opportunities and built both active and passive income streams. Building passive income streams as a freelance Java developer is not easy. It takes lots of time, dedication, and creativity. It is not for everyone. If you are one of those few dedicated Java developers who want to explore and expand your horizons as a Java developer, then read on. It won't make you a millionaire, but it can provide you with descent income streams and variety of experiences and opportunities.

Here is a pictorial representation as to what you can do as a Java developer or Java freelancerto open more doors.

Motivated, skilled, and networked professionals with good soft skills, marketing skills, business focus and right attitude may be the ones least affected by economic down turns, but even they are not guaranteed to survive in this volatile industry. Companies are more keen on balancing the budget than worrying too much about how much knowledge, skills, and capabilities walk out the door. At times, it really matters to be at the right place at the right time or know the right person at the right time. So, how do you cater for these types of uncertainties in your career? This can be done via networking and creating additional streams of income, even if you are an employee. There are a number of streams one can look at:

Finding a side job as a freelancer via http://www.elance.com/, http://odesk.com/, http://www.guru.com, etc, part-time job as a teacher, an instructor, or a lecturer at your local training institute, technical college or university, offering part-time consulting services in your field of expertise through networking. But check your contract to see if it allows any side job. The contract jobs are different from freelancing work. Freelancing tends to be on a per-assignment basis, while other contract options could be on a monthly/daily/weekly rate for performing certain services.

Starting an ad-supported blog – e.g. Wordpress, Blogger, etc with Google Adsense, WordpressAds, and other affiliate programs. Firstly, build an interesting and informative blog before start advertising. Build up a reasonable size followers before including advertisements. Learn the SEO (i.e. Search Engine Optimization) techniques. If you have enough visitors to your site, sign-up for Google adsense to earn some extra income. You could have a single blog earning $2 - $40.00/day or 10 different blogs earning $1.0 - $20.0 /day. Explore the other advertising avenues and affiliate programs (e.g. Amazon.com affiliate program). If you can get around 50,000 page views, you can earn around $30.00 to $50.00 with advertisements. You need at least 30-40 quality posts to start with, and get "Google analytics" to see what posts are doing well, where the traffic is coming from?, etc. There are myriad of online resources for SEO techniques to drive more traffic. It is a very small incentive while you are learning, capturing your experience, networking, and marketing yourself.

Writing your own book. Publishing has never been easier and very affordable through the print on demand (aka POD) publishing like lulu.com, iUniverse.com, Lightningsource, createspace, etc. The blog entries you had created over a period can be converted into a book.

Other avenues your creative mind can think of in terms of new products, applications, etc. A colleague of mine was working full-time as a software engineer and created an online application for "tennis court booking system" on a part-time basis. He basically combined his passion for playing tennis and writing software. Another example would be to build iPhone applications by getting a mac, SDK for iPhone development, learning objective C, and start writing. Alternatively, if you have a great idea, hire an iPhone developer to implement your idea. The idea is more important.

Additional streams may not -- at least initially -- make enough money to replace a full-time job, but they can cushion the pain of income loss while helping you acquire new skills, keeping positive and motivated, and growing your professional network. Publishing books/articles (http://java.dzone.com/ ) and writing blogs can also build a professional profile online. The Web has become a very valuable tool for communicating your value as a professional, expressing your capabilities, networking (e.g. LinkedIn ), etc. You don’t have to be a jack of all trades or a brilliant professional to build your own brand online. All what takes is to be creative and express your unique value with passion and perseverance. If things are looking more promising, you can promote your online products, books and services through your blogs and web sites. So put your thinking cap on and see where it takes you.

Don't let additional income let you lose focus

While it is encouraging to have additional income streams, it is imperative to put things into perspective. For many, employment will be the main source of income. The key focus must be on

If you are a beginner, acquiring the much needed hands-on experience via voluntary work, self-taught, projects, open-source experience, etc and creating an online portfolio to promote your skills and passion will be a better choice than going down the path of enhancing your academic qualifications through post-graduate degree or certification. The prospective employers are more interested in experience.

If you are a blogger, then pay more attention to writing quality and unique blogs that will be useful to you and others. Quality blogs can impress your potential employers, help you network with like minded professionals, help you capture your thought bubbles and experience, and conducive to turning into a book.

"Logic gets you from A to B. IMAGINATION will take you everywhere" -- Albert Einstein

If you are serious about expanding your horizons, check my new book entitled "How to open more doors as a software engineer?".

Apr 17, 2012

Multi-threading with blocking queues

Q. How would you go about implementing a scenario where two individual threads generate odd and even numbers respectively, and a third thread that prints the sum of odd and even number that is a multiple of 5

A. This can be achieved with multi-threading and a BlockingQueue. Earlier we saw this with piped reader/writer. A queue is a First-In-First-Out (FIFO) data structure.

A blocking queue has the following behavior

A thread trying to take from an empty queue is blocked until some other thread puts an item into the queue.

A thread trying to put an item in a full queue is blocked until some other thread makes space in the queue, either by taking one or more items or completely clearing the queue.

Apr 4, 2012

How to become a software architect?

In industry specific forums, I often see questions like “what certification do I need to do to become an architect?”. The simple answer is that you don't need a certification to become an architect.It may help, but there is a lot more to it to become an architect.

"If you need to be successful in anything, you need to emulate those who are already successful"

You can't just only study to become an architect. The best way to become an architect is to start thinking and acting like one. If you start thinking like one, you will start to observe the architects and learn. This is true for the other roles like becoming a manager, lead, etc. These roles require technical skills complemented with good soft skills, attitude, and work ethics.

Any self-help book will tell you -- what you think and do is what you become. This is why many interviewers ask open-ended questions like who are your role models? what books did you read recently? how do you keep your knowledge up to date? tell me about yourself? what are your short term and long term goals?, etc. These questions can reveal a lot about your passion, enthusiasm, attitude, communication skills, and technical strength.

Here is my take on the road map to become a software (e.g. JEE) architect.

Get a good handle on the 16 key areas and proactively apply these key areas.

Language Fundamentals (LF)

Specification Fundamentals (SF)

Platform Fundamentals (PF)

Design Considerations (DC)

Design Patterns (DP)

Concurrency Management (CM)

Performance Considerations (PC)

Memory/Resource Considerations (MC)

Transaction Management (TM)

Security (SE)

Scalability (SC)

Best Practices (BP)

Coding (CO)

Exception Handling (EH)

Software Development Processes (SDP)

Quality of Service (QoS)

Look at things from both business and technical perspective: architects form a bridge between many cross functional teams like business analysts, stake holders, project managers, developers, testers, infrastructure team, operational and support staff. Know your target audience and learn to convey technology to the business non-technically and the business requirements to the techies technically.

Learn to look at the big pictures and also pay attention to details where required.

Get a well rounded hands-on experience. For example, client side, server side, application integration, full SDLC, etc. Nothing beats experience, and you can proactively fast-track your career by learning from others' experience via good books, blogs, industry specific web sites, and helping others on the forums.

Also, should have a good domain knowledge

You don't have to be the "jack of all trades" but as a technical leader and a bridge between various stake holders and the development teams, you need to have good soft skills to make things happen by engaging the right teams and expertise. The key soft skills to have are communication, interpersonal, leadership, analytical, negotiation, and problem solving. The soft-skills and domain knowledge are the most important in the list.

So, the combination of all the above can transform you into an architect. Stay visible at your current work and behave like an architect to get a foot in the door. Ask the right questions and contribute in team meetings and crisis sessions to prove your capabilities.

The best way to do this is to write some basic code and then have a pictorial representation as to how the objects are created, how do threads work and communicate with each other, and what is stored in a heap and what is stored in a stack, etc.

Here is an example of a producer thread (thread-0) producing by incrementing the counter from 0, and the consumer thread (i.e. thread-1) consumes by decrementng the counter. These two user created threads are spawned by the main thread, which is created by the JVM and is alwyas there by default. The ConsumerProducer is the shared object with synchronized methods that commuincate with each other via wait( ) and notifyAll methods. Only one of the two synchronized methods can be executed at a time. The wait( ) call in consume( ) relinquishes the lock to the produce( ) method, and once the produce method has incremented the count, it notifies all threads and one of the waiting threads will resume. In this example, there is only one waiting consumer (i.e. Thread-1) thread. So, both threads will be communicating with each other via the wait( ) and notifyAll( ) calls in the shared object ConsumerProducer. This is an example of the producer-consumer design pattern.

Firstly, look at the code and then the diagram. The diagram is simplified to get an understanding and should not be construed as exactly what happens in the JVM.

The main thread spawns a consumer and a producer thread. The ConsumerProducer is shared between two threads. The boolean flag is used to signal if it is a consumer thread or a producer thread to invoke the relevant methods.

The JVM is a process. As you could see in the diagram above that each thread has its own stack, but share the same heap space. One heap space per process.

The local variables like producer, cp, and consumer are stored in the stack along with the method calls like main( ), run( ), etc.

The objects are created in the heap. For example, 2 instances of ConsumerProducerTest and a single instance of ConsumerProducer. Unless the methods of these instances are properly managed, multiple threads can concurrently access these intsnaces to cause thread-safety issues.

There are scenarios where the threads want to commuincate with each other. For example, a worker thread produces something and a consumer thread consumes what the producer thread consumed. This can be achieved through thread coordination with the wait and notify( ) / notifyAll( ) methods. In the above example, since both the produce( ) and consume( ) methods of the object ConsumerProducer are synchronized, only one thread can execute either one of the methods. If the count == 0, the consumer thread relinquishes the lock by invoking the wait( ) method and waits for it to be notified again to resume. The producer thread will increment the counter and then will notify all the waiting threads via the notifyAll( ) method. One of the waiting threads will then be able to resume from its waiting state.

The local variable isConsumer is used to differentiate between a consumer thread and a producer thread.

The count is an instance variable shared by both the consumer and producer thread in a thread-safe manner.

Here are the points that answers the above questions.

Each time an object is created in Java it goes into the area of memory known as heap.

The primitive variables like int, double, etc are allocated in the stack if they are local variables and in the heap if they are instance variables (i.e. fields of a class).

Java is a stack based language and local variables are pushed into stack when a method is invoked and stack pointer is decremented when a method call is completed.

In a multi-threaded application, each thread will have its own stack but will share the same heap. This is why care should be taken in your code to avoid any concurrent access issues in the heap space.

The stack is thread-safe because each thread will have its own stack with say 1MB RAM allocated for each thread but the heap is not thread-safe unless guarded with synchronization through your code. The stack space can be increased with the –Xss option.

All Java methods are automatically re-entrant. It means that several threads can be executing the same method at once, each with its own copy of the local variables.

A Java method may call itself without needing any special declarations. This is known as a recursive method call. Given enough stack space, recursive method calls are perfectly valid in Java though it is tough to debug. Recursive methods are useful in removing iterations from many sorts of algorithms. All recursive functions are re-entrant but not all re-entrant functions are recursive.

Idempotent methods are methods, which are written in such a way that repeated calls to the same method with the same arguments yield same results.

Each time an object is created in Java, it goes into the area of memory known as heap. The Java heap is called the garbage collectable heap. The garbage collection cannot be forced. The garbage collector runs in low memory situations. When it runs, it releases the memory allocated by an unreachable object. The garbage collector runs on a low priority daemon (i.e. background thread). You can nicely ask the garbage collector to collect garbage by calling System.gc( ) but you can’t force it.

An object’s life has no meaning unless something has reference to it. If you can’t reach it then you can’t ask it to do anything. Then the object becomes unreachable and the garbage collector will figure it out. Java automatically collects all the unreachable objects periodically and releases the memory consumed by those unreachable objects to be used by the future reachable objects.

If two objects have circular reference with each other in the heap, but none of them are reachable from any thread, then those circular referenced objects can be garbage collected.

Apr 2, 2012

Java stack data structure interview questions and answers

The Java Collection library is one of the most frequently used libraries.The LIFO access mechanism used by a stack has many practical uses. For example, Evaluation of expressions / syntax Parsing, validating and parsing XML, undo sequence in a text editor, pages visited history in a web browser, etc. Here are a few Java based interview questions and answers on a stack.

Q. What LIFO implementation would you use in Java?A. The Vector based Stack class is a legacy and the Javadoc documentation for this class states that Deque should be used instead because it provides better LIFO operation support. Implementations of Deque are typically expected to perform better than Stack. J2SE 5 introduced a Queue interface and the Deque interface extends this interface. The Deque interface supports should be pronounced "deck" rather than "de-queue" and is not related to dequeue used to indicate removal from a queue. The double-ended queue supports addition or removal of elements from either end of the data structure, so it can be used as a queue (first-in-first-out/FIFO) or as a stack (last-in-first-out/LIFO).

Note: The legacy Vector class uses internal synchronization, and it is rarely good enough for actual consistency, and can slow down execution when it is not really needed. The new java.util.concurrent package provides a more efficient way of thread-safety.

Q. Can you write a program to evaluate if a given string input has proper closing bracket for every opening bracket? A. Firstly, think of the pseudo code. The pseudo-code goes as follows.

1. Store every opening parenthesis (i.e. a LHS parenthesis) in a stack. This will enable LIFO.
2. When you encounter a closing parenthesis (i.e. RHS parenthesis ), pop the last entry, which should be the corresponding opening parenthesis.
3. If not found, then the parentheses are not matched.

If required, draw a diagram as shown below.

/opt2/ETL/working/MAS_WRAP_BDT_DEV/script

Here is a sample program to illustrate a Stack (i.e. LIFO) in action in evaluating if a program has balanced parentheses. The enum constants class that defines the parentheses.

Now, the stack in action using its LIFO mechanism to marry a closing parenthesis (i.e RHS) with an opening parenthesis (i.e. LHS). If you find any LHS parenthesis, push it into a stack, and when you find a RHS parenthesis, pop the stack to see if you have a corresponding LHS parenthesis.

Note: The above example is shown to illustrate how a LIFO mechanism can be used to determine if the parentheses are balanced. The actual implementation is far from being optimum. Where ever there is a large block of if-else or switch statements, one should think about an object oriented solution.

Q. Why Deque interface is different from other collection classes? A. In Deque, You can insert and delete the objects from the both start and end of the the collection. Whereas in a normal collection, inserts/deletes are happening at last only.

Q. What are the different ways to look at the trace of your program execution?A.

1. Java is a stack based language, and the program execution is pushed and popped out of a stack. When a method is entered into, it is pushed into a stack, and when that method invokes many other methods, they are pushed into the stack in the order in which they are executed. As each method completes its execution, they are popped out of the stack in the LIFO order. Say methodA( ) invoked methodB( ), and methodB( ) invoked methodC ( ), when execution of methodC( ) is completed, it is popped out first, and then followed by methodB( ) and then methodA( ). When an exception is thrown at any point, a stack trace is printed for you to be able to find where the issue is.

2. A Java developer can access a stack trace at any time. One way to do this is to call

Thread.currentThread().getStackTrace() ; //handy for tracing

You could get a stack trace of all the threads using the Java utilities such as jstack, JConsole or by sending a kill -quit signal (on a Posix operating system) or <ctrl><break> on WIN32 platform to get a thread dump. You could also use the JMX API as shown below. ThreadMXBean is the management interface for the thread system of the JVM.

Q. Is recursive method calls possible in Java?A. Yes. Java is stack based and because of its LIFO (Last In First Out) property, it remembers its 'caller'. So it knows whom to return when the method has to return.

Q. How would you go about analyzing stack traces correctly?A.

1. One of the most important concepts of correctly understanding a stack trace is to recognize that it lists the execution path in reverse chronological order from most recent operation to earliest operation. That is, it is LIFO.

2. The stack trace below is simple and it tells you that the root cause is a NullPointerException on ClassC line 16. So you look at the top most class.

Exception in thread "main" java.lang.NullPointerException
at com.myapp.ClassC.methodC(ClassC.java:16)
at com.myapp.ClassB.methodB(ClassB.java:25)
at com.myapp.ClassA.main(ClassA.java:14)

3. The stack trace can get more complex with multiple "caused by" clauses, and in this case you usually look at the bottom most "caused by". For example,

Exception in thread "main" java.lang.IllegalStateException: ClassC has a null property
at com.myapp.ClassC.methodC(ClassC.java:16)
at com.myapp.ClassB.methodB(ClassB.java:25)
at com.myapp.ClassA.main(ClassA.java:14)
Caused by: com.myapp.MyAppValidationException
at com.myapp.ClassB.methodB(ClassB.java:25)
at com.myapp.ClassC.methodC(ClassC.java:16)
... 1 more
Caused by: java.lang.NullPointerException
at com.myapp.ClassC.methodC(ClassC.java:16)
... 1 more

The root cause is the last "caused by", which is a NullPointerExceptionon ClassC line 16

4. When you use plethora of third-party libraries like Spring, Hibernate, etc, your stack trace's "caused by" can really grow and you need to look at the bottom most "caused by" that has the package relevant to you application like com.myapp.ClassC and skip library specific ones like org.hibernate.exception.*.

Q. Can you reverse the following numbers {1, 4, 6, 7, 8, 9}?A. There are a number of ways to achieve this. Speaking of LIFO, the following example illustrates using a stack based implementation.