I can't stand the them and us attitudes that seem pretty prevalent about automation engineers versus manual testers. You hear things like:
- automation engineers are hopeless testers - give me a manual tester any time
- manual testers of course can't possibly code - give me a programmer
Of course, there are some people who perhaps can only do one of those things well, but I believe many can do both well with practice. I like to think of automation co-existing with manual testing and both augmenting the other.
The belief that automation only finds the same bugs misses the automation process. Yes, I agree, if you just run the scripts, then the same code paths will be exercised and the same checks will be done, but the process of writing automation scripts isn't as simple as that.
The fact is, as the System Under Test (SUT) changes and new releases are put into the test environment, the automation scripts run, and due to changes in the GUI and the work flow, the tests will often 'break' - this may be due to a bug being found - great - and we have found it fast. Often however, it won't be due to a bug - it will be a 'bug' in the automation script - or to be more precise, the automation script isn't flexible enough to cope with the change to the SUT. So what happens?
Well, the automation engineer will start to investigate. He or she will probably manually look at the SUT and or step through the automation script. So, he will start to focus on the part of the SUT which has changed (and therefore an area where a real bug may have been introduced).
Another area where automation engineers find bugs is in the process of writing a script for the first time:
Often the automation engineers won't be overly familiar with the SUT - this is a good thing - as a fresh pair of eyes I believe can often find unexpected bugs - the new pair of eyes doesn't exercise the SUT in the expected well trodden path - they can act as a 'monkey' test. Just last week (using TestComplete), I was automating a new test on some new software (to me) and found a bug - the automation script found the bug, but it was my manual 'testing' - I was just playing with the system really to try and understand how to stop a process through the GUI - that had broken the SUT. The web app had to be restarted to 'fix' the SUT - quite a serious bug - and one that hadn't been found to date by manual testing.
And finally, the key thing for me is that automation testing takes a massive burden off of the shoulders of the manual testers, who no longer have to plough through boring manual scripts making lots of error prone checks. The automation can do that and they are freed up to do higher value exploratory testing.
I dont think anyone has been making the argument that automation frees up testers to do the things they are good at - have they ?
ReplyDeleteI also havent heard anyone say that automation engineers are useless testers or that manual testers cant code - sources ?
I'm not too convinced about your automation finding bugs argument where you say that automation breaks, the engineer looks into why and finds a bug. Try making that argument - "the automation will fail but it will be a false negative but when we look into it we might find a bug not connected to it"
Same applies to automation engineers not being familiar with the SUT - if they are not familiar with it how can they write good tests ?
Hi Phil - thanks for your comments.
DeleteI guess I am making the argument that automation frees up manual testers (as one of the many benefits) - I'm surprised if others haven't been making that argument.
I don't have sources for these statements, though I am sure I could go and find some views such as this on the web - these are just the views I pick up when I meet people during the course of my work.
I am an automation engineer predominantly and the post was supposed to be (mainly) about proclaiming some of the benefits of automation. I was trying to emphasise the automation process as I see it and how automation engineers find bugs often using a mixture of automation and manual testing.
I also find often automation engineers are brought in 'from outside' and often don't have a great deal of knowledge about the SUT. They are often given testcases to automate. It is up to the automation engineer to use these testcases and to us others sources such that they can write good tests - for example - talk to other testers, developers, BA's etc etc....
I'm happy to find numerous useful info here in the post. I would really like to come back again right here for likewise good articles or blog posts. Thanks for sharing...
ReplyDeletetop engineering colleges in pune
pune engineering colleges
Thanks!
DeleteTony, I am really curious to the context in which this post came to be. What kind of confrontation have you had that triggered this? Would you mind sharing the background with us?
ReplyDeleteI agree with you, there should be no us and them, especially not between testers. Test automation is there to make manual testers life easier and less repetitive.
Hi Martijn - I'm glad you agree with some of what I had to say.
DeleteNo one major event triggered the post - I just wanted to 'defend' automation in particular - I come across quite a lot of negative sentiment (and lots of positives too to be fair), but I just wanted to try and bring out some of the 'softer' benefits of the automation process as I see it. I am also aware automation is not a 'silver bullet' - it is just another arrow to our bow as testers/software engineers.