For many companies Automation testing is an attractive term, they say “We have automated our application up to 80% and reduced the manual effort tremendously”, some big companies like Microsoft says, we have automated the Windows 8 mobile OS and tested it for 10 million hours before release, it’s really an catchy quotes and almost all companies badly need this, but the truth is, while it comes to reality all this quotes
fades out.
The real problem is not with the automation testing tools or the process, it’s how the companies are looking at the automation itself, the automation should come hands on hand along with application development for long term rather short term.
Let’s discuss first about the short term automation testing nature of application
Short-term automation
Since most of the Service based/Product based (Agile) companies are now focusing on automation testing to reduce their manual effort, they start doing something like this.
- Choosing a best available tool( Open source/Proprietary)
- Creating a framework (Or reusing what they have)
- Start writing test cases for UI of application
- Executing and maintaining the test cases.
But for every release when there is a UI change in application, again they do the following
- Modifying their framework a little bit to adapt new changes
- Modifying the Test cases for UI (Since its changed)
- Executing and maintaining test cases
As you could see above there are lot of repeatable “baby sitting” operations, which an automation engineer has to do on daily basis, life of automation will be happy until he starts to write a framework and make complete (at least 80%) application automated, but before a take a sigh of happiness, the UI of application changes and his work starts once again.
The problem is, many Service or Product based companies are focused on resolving the day to day problem using automation testing for “Short term”, investing less in automation, than seeing the big picture of real automation, in fact, UI based automation is “NOT” going to reap the benefit as the application is more rapidly changing to support different kinds of devices.
So what is the best way to automate a rapid changing application?
How to overcome all the above sited issues?
What possible steps we can take to make automation engineer feel more comfortable?
To answer all the above question, the answer is very simple, see the big picture of automation, consider automation as a “
Project” which should come hands in hands with both Development and testing of product, and set a long term project plan for automation along with application development life cycle, so that, any release of product should come along with automation testing coverage.
Long term automation
For long term automation testing to happen, following has to be taken in consideration during design phase of product itself
- Whether my product is going to be fully automation tested?
- Are we going to rollout product along with automation test cases?
- How to define an object in application, which will seldom change.
- Create a complete environment for automation (Automation Lab) which should never be used by any other person other than automation test engineers.
If above all is done, so what is the tradeoff?
Consider that the benefit of automation is already 40% achieved.
The above shown is Long term goal of automation testing plan, well what is the long term goal of automation itself?
- Creating a framework which should support Unit testing, Integration testing and UI testing of application
- Implementing test harness to support automated installation, deployment and Sanity/Regression testing of application.
- Implementing drilled down reporting which should give nitty-gritty detail of application
So, what is the percentage of each test execution should be done
As you could see above, the UI testing starts to reduce from 40% to 15%, but the Unit testing of application has increased from 15% to whooping 70%, which clearly indicates that, automation testing in any company should focus on Unit testing rather the
fragile UI testing itself, so that the benefit reaped out of the automation is higher and long term.
Well how?
As the application is transforming to support multiple devices as opposed to just one single device in early 2000’s which was just Personal computer, now people are interested to work on any device ranging from cell phone to tablet and from table to PC’s.
Which means the application will change for each and every application, but the “
Core” applications execution engine remains the same, and that’s where Unit testing has to be taken care as the most important priority.
To summaries, for any Long term automation project, UI automation should be taken in least consideration, but the applications core execution engine (Unit testing) should be taken in higher priority, which needs to be fully tested. This will make the automation test engineers life more comfortable and even more challenging.
Please leave your comments to improve the post
Thanks,
Karthik KK
As you could see above, the UI testing starts to reduce from 40% to 15%, but the Unit testing of application has increased from 15% to whooping 70%, which clearly indicates that, automation testing in any company should focus on Unit testing rather the fragile UI testing itself, so that the benefit reaped out of the automation is higher and long term.
Well how?
As the application is transforming to support multiple devices as opposed to just one single device in early 2000’s which was just Personal computer, now people are interested to work on any device ranging from cell phone to tablet and from table to PC’s.
Which means the application will change for each and every application, but the “Core” applications execution engine remains the same, and that’s where Unit testing has to be taken care as the most important priority.
To summaries, for any Long term automation project, UI automation should be taken in least consideration, but the applications core execution engine (Unit testing) should be taken in higher priority, which needs to be fully tested. This will make the automation test engineers life more comfortable and even more challenging.
Please leave your comments to improve the post
Thanks,
Karthik KK
