Blog Stats

Category: Automation

Most software were developed by highly skilled programmers. But still none of them are bug free and many of them are very buggy.

“If builders built buildings the way programmers write programs, the first woodpecker that came along would destroy civilization.” – Jerry Weinberg

Software is a very volatile product, so it’s practically impossible to build bug free software even by the best of the programmers. So there exist testers to test the product. Testers look into a product with different perspective to inform management about the health of the product, thus helping management to take decisions better and faster. Most releases will go smooth and there will be failures in between (or vice versa 😉 ). Everything looks awesome until something goes wrong, then everything comes under the scanner. There will be a high-profile meeting to discover that THE problem was testing not done properly and THE reason was lack of time and THE solution is automation. 🙂

Now some testers who had talked about automation earlier will become automation test coders. These testers will evaluate automation tools and select the tool that has more voting in a polling conducted by ***interviews.com forum. A framework that Google shows first would be chosen as the best suited one. A PoC that automates the login page of their application. Now stage is set to automate 90% of test cases. NOTE: 100% is not possible!!!

This is something similar to a sip test and home use test of an edible products, there was an interesting analysis about the failure of New coke in the book Blink. I too had many snap judgment failures and one among them was selecting a gaming console. In one of my earlier jobs where I was a game tester, was given an option to join either Xbox or PS2 team on my comfort with the consoles. I tried both for about ten minutes and I felt comfortable with Xbox, so moved to Xbox team. But later when I played for more than 2 hours continuously, I realized it is not comfortable at all, later found PS2 was more comfortable for longer time duration.

Without anticipating all such problems, a set of testers who were either bored by the kind of testing they do or misguided about automation testing, would become an automation engineer without much programming skills. They would then write buggy codes to find bugs in an application that was developed by skilled programmers , read the blog title again. 😀

In Kung Fu Panda II, Po finally stops the thing that stops kung Fu after finding a secret . The secret here is to invest in developing thinking, analytical skills of testers not just on automation tool.

Automation can help to solve the problem, but that is not the solution for all problems.

PS : kung fu panda 2 – An abandoned villain threatens, there exists an unknown secret, Po sets out a mission to understand the secret to defeat the villain and bring peace. It is twice filled with attractiveness and awesomeness …

Advertisements

Share this:

Like this:

This is Manual Locker 🙂 . He takes 2 to 3 seconds to lock and almost the same time to unlock. Once you pull him to left you can clearly see that he has locked the door and you feel very secure. Purpose Solved

And this is hi-fi one click automation locker 🙂 this guy needs just one click to lock. But do you ever feel secure after using this lock? At least I am not. I always test this lock first by locking it with doors kept open and try to open it from outside to make sure the lock works 😀

The same is true with manual and automation testing. How many of us feel comfortable after the so called automation scripts run and say the script passed?

What other benefits does automation lock provides than manual lock? I don’t think anything great in this context.

Both the locks solve the same purpose with no added benefits by automation. Yes, it may save 1 to 2 seconds, but considering the resources and time invested to design and manufacture the automation lock, was it worth investing? I doubt

Not convinced? Read further to know what happens if automation lock fails miserably. One sad day my friend got stuck inside bathroom for close to one hour when the critical spring inside the lock broke. So there was no option other than breaking the lock to release him :D. Purpose dissolved

Ok, now take a look at the amount of resource wasted. Also wondering how much time wasted to design such a fancy lock. At the same time the manual lock is very simple, uses less resource, simple design and solves the desired purpose neatly.

This is how automation testing done at most places, if your manual testing solves the purpose what is the need for automation testing?

Resource utilization, if there is not much testing due to delay in development put them in automation

“Succumbing to The Golden Elephant Syndrome: James Bach calls one of the terrible pathologies of testing. A white elephant may be a big, useless thing, but if it’s made of gold and costs a lot of money, it’s tempting to some people to try to use it anyway. Expensive tools can cause a lot of trouble if they are badly designed and unreliable. If such a tool were cheap, we wouldn’t hesitate to throw it away. But if it’s expensive, the person who bought it doesn’t want to look like a fool-and thus becomes a bigger fool “ – this is an extract from Perfect Software:And Other Illusions about Testing

Think again!

Instead, why don’t we allow the testers to explore, question the product, so they might uncover some critical bugs, which client might discover later. Such exploration might bring in new ideas, suggestions and enhancements to the product that might help to better your competitors. Encourage testers to improve their critical thinking and analytical skill .Conduct bug battles and encourage them to write articles, blogs at least within the organization.

From my experience everywhere regression automation is a fancy way to attract clients. For instance if you goal is to reach the top floor you may take elevator instead of staircase, so you reach quicker. But at most place goal is to walk but most try to use elevator to solve the purpose.

Most of us on seeing such attractive advertisements go to the sale with lot of planning and enthusiasm to get many items cheaper. But on entering the shop any item we pick will always have zero or very minimal off plus lot of conditions attached to it. [Huh, now you realize that the advertisement says UPTO80% and a tiny astrix attached to it]

Similarly GUI automation tool advertisement also say it can reduce regression testing cost by up to 80% or such similar tag lines, just by showing a login page automation. I was shown with the similar login screen example in almost every session, demos or presentation I participated. Mostly it goes like this

The presenter would ask to give the login screen of our application. The he would record a script to input the login details, after a successful login he validates the home page. Now he will go and alter the script so that it validates for incorrect login details. Then he shows data parametrization options for many data and proudly says we have automated so many scenarios for a login page in no time. Unfortunately, managers who do not have automation experience were made to think that automation could be so simple and will recommend such GUI tools. Because they see something like 80% cost reduction at that point.

But the first thing I do on seeing such demo is to enable remember password option and then run the script, now the script fails because it can’t find the login page. Presenters will mostly reply that we need to apply some conditional statements to handle that (Now did you get the first condition applied?).Fine, then my application opens a Change Password page on password expiry, the script fails because the script always checks for home page after login. Presenter will think for a while and would say that we need to handle this with some conditional statements. (Now did you see the second condition applied?)Fine, then my application some time gives a full page banner about new announcements or new feature added, like how our online banking starts displaying investments plans or new service offers. Yes you readers got it correctly this is the third Condition applied. Manager also by this would have realized that the offer is not really worth 80% he might recalculated to 60%.

Next, here is another big question I ask, what if all the three of the above appears or two of them appear at any given point of time. Such cases we need to consider the order in which they appear also. Can we automate them in no time?

No, we can’t automate them just like that. We need to write functions, conditions checks and also loops, realize that this is getting into kind of programming by itself. So we now not only write the script but need to be tested as well. [We all know very well that as per D’cruz Defecto Injection law (DDIL) bug count increases exponentially for every conditional block coded. Hey wait don’t Google for DDIL because it do not exist. Then wondering why I mentioned that, because I want you guys to read Pradeep’s post Black Viper Testing]

Now calculate how many such conditions would need to be handled while automating the entire application. Manager realizes that he definitely needs to rethink on the offer that the tool will return. Now did you realize that the festival sales offer and GUI automation tools offer are the same? But wait I got one difference between them, in festival sale Up To 80% means 0 to 80% but in automation this can even go negative 🙂

A good amount of Regression Checks could be automated, but that need a solid approach, planning and good investment (both time and resource). Successful automation can’t be done as a sideline activity with testers who don’t have technical or programming skills.

Share this:

Like this:

Smart identification is way by which QTP identifies an object when it is not found in the object repository during the run session. But it is not advisable to use this feature. Because QTP assumes and finds the object during run session, it is not a good idea to assume any think in software testing.

Let us see a simple example how QTP smart identification fails.

Scenario

I want to verify a simple web page which comes, says after I perform some transactions in a website. The web page has a simple text message “Your transaction is successful” after a successful transaction and “Your transaction is not successful” after an unsuccessful transaction. Here you are automating a scenario to check a successful transaction page exists or not after a successful transaction.

The code to check that is

If browser (“Smart is not Smart”).Page (“Smart is not Smart”).WebElement(“Your transaction is successful”).Exist Then

reporter.ReportEvent micPass,”Pass”,”Result Pass”

Else

reporter.ReportEvent micFail,”Fail”,”Result Fail”

End If

This is the sample of that page

And the result is

But now consider that the transaction is not successful and you got a message “Your transaction is not successful”

Here is the sample page

But if you run the above code, the result will be still passing.

What cause this problem? Mr. Smart Identification

The only difference between those web element in successful and unsuccessful transaction is the text. So QTP assumes both the object is same and says pass. Yes, it gives a warning on using smart identification but it is visible only in the QTP result window. If you are using some other external means to document the result, this will never get noticed.

So it is always better to disable smart identification, the simplest way to do is go to File –>Setting–>Run tab and check “Disable Smart Identification during Run Session”

If you want to disable smart identification in you Quicktest Automation Object Model, the script is

Share this:

Like this:

This is an attempt to write a sequence of articles under“What Matters Most?”

This is to share my experience, how different thinks matters in the field of software testing to bring effective and efficient solutions.

Here is first such an attempt to see how Focusmatters.

Of late, I am very addictive to Farmville, a small interesting game in Facebook. Let me brief about this game. This is a simple and fun game where you do farming, add your facebook friends as neighbors, and move up in the game. You can send gifts to your neighbors from the list of available gifts. You need to spend only 5 to 10 minutes a day to make most fun out of this game. It is easy to send same gift to all the neighbors. But as my neighbor list grows, and as each of them wants a different gift it takes longer time to send them the gifts. Suddenly the automation tester within me woke up and said “automate this” (he wakes up very rarely 😉 though). I said wow! That’s a nice idea, let me attempt this and within half an hour, I got a very effective solution. The idea is

1. I need to get the entire request from friends and store it in a single file.

Solution:Oh! There is Google, ask all my friends to update the worksheet in Google Docs, with their wish list

2. Now read this file and start sending their gifts

Solution: Any automation tool can do this by reading the worksheet and schedule this run every day

Now base solution (no no I am not talking chemistry here 🙂 ) is ready, start building on this by adding all the other features like

a. What if gift asked by the neighbor is locked? (The list of gifts gets unlocked one by one as you move up in the game)

b. Send a mail back to neighbor saying that the gift you asked is locked

c. Club similar gift and send them in a single attempt (improve efficiency 🙂 )

This made me to wonder, how did I came up with such a solution so quick? This rarely happens in my projects 😉 . So I started analyzing how I got this solution?

It is “Focus” that made this possible. I just focused on one think need to automate this. I did not think too much on other factors like what tool to use? , Schedule, resources, etc. Of course, all those are important but focus on them later, to get the solution just focus on one goal at a time (literally goal as in football, can be scored only one at a time 🙂 ). To emphasis more on this, my initial focus on just automating and not concentration on other factors naturally helped me to get other solutions as well.

So, Focus on Mission

Happy Focusing! (I should thank Pari, for adding such greetings at the end of the articles)