Incremental Automation

When you're automating, it may seem logical to automate an entire test plan before you begin running it. For example, if you're automating a smoke test, you may want to have all pieces of the smoke test scripted so that you can run the test at the touch of a button. That's a good goal to strive for, but that doesn't mean you can't automate a piece of that smoke test and then start running it right away.

Back when I was first learning to automate, my smoke test was a 26 page test plan that touched all the major features of our product. When I sat down to automate it, I started on page 1 and went step by step. I was doing great until I hit page 11, when a control in my AUT wasn't recognized by my test tool. I spent a couple of days tracking down the issue and coming up with a workaround. A similar problem happened later in the test. All in all, it took about 3 days to automate the test, and another 3 days to troubleshoot issues, so it was over a week before the script ever ran. This was my first experience with automation, so I thought this was probably normal.

Looking back, I can see that by not running the parts of the test that were finished, I lost a great opportunity. I could have had the first 11 pages of that test running while I troubleshot the issues. Then people would have seen that the automation was doing something, and I could've shown my manager that while I was encountering some issues, our first crack at automation was off to a good start. I've heard stories where people tried to automate an entire test, and due to one problem or another, didn't actually run the test until two or three days before their product was set to release. That sort of approach doesn't benefit anyone, and winds up costing time and effort instead of saving it. It also reduces automation's credibility in management's eyes, which is part of the reason (I think, anyway) of why so many automation efforts get abandoned.

So the moral of the story is don't do "big bang automation", where you wait until everything is finished and perfect before you start running your automated tests. Get a piece of your test automated and then start running it. Then continue adding pieces until your entire test or suite of tests are finished. Your team will see results faster from automation, and your management will be more confident that the automation is helping the developing efforts.