At first, we should address the following questions before we consider how to implement automation test together with manual test:
- Automation test: Will it automatable for the testing work? What level of automation should be done? Do we have sufficient and skilled resources for automation? Do we have enough time to automate necessary test cases? etc…
- Manual test: Do we have sufficient manual testers? Do we have enough time to complete required test tasks? What is the test priority? How to maintain a good relationship with developers, etc..
Then, I will use a real case to explain how did we do to use the automation test and manual test within one project. My latest project is function test for a financial software, the automation is about 20%and mainly used in BVT test, system test and regression test.
BVT test, it uses a small case set to cover the basic functions in order to quickly ensure the product can be used in the next stage without critical issue. Automation is very useful.
System test, the data is confirmed, the process is confirmed, for a mature financial product, Automation only can verified one set of calculation is right, but as all we know, there are many complicated calculation process for one feature. So In system test, our goal is only to assure that the function of product can be succeed in one or two process. Base on this goal, Automation is enough during this test period. In manual part, we just need to prepare the test data and the test case with at least one test process. Then manual team has the time to understand and familiar with the next release.
For the financial product, most of features have the complicated calculation, if we only test them by manual test, we’ll face two challenges:
- 1) there is no enough time to cover all calculations in testing;
- 2) complicated calculation make the tester spend much time and energies to confirm the accurate arithmetic.
So in my project, we summarized and created the automation scripts for all calculations, then during the regression test, we can use the Automation to do the calculation test directly. Even during the new function test period, automation can help us to do the related calculation test, then testers have time to do the ad-hoc test and find some deeper bugs.
I believe many of you are working in manual and/or automation testing field. Though I have addressed my practice in above articles, I still want to hear your views in handling such kind of software testing challenges. Feel free to share your thinking.
