Focus on Test Approach, Not Test Tools

There's a lot of news flying around right now about how powerpoint is making us stupid. Thing is, I don't think this is a powerpoint problem, I think it's a people problem. Slides probably aren't the best method to communicate this information, and that certainly needs to be remedied. So it's the approach that these presenters are taking that's the problem, not powerpoint itself.

This got me thinking about some of the negativity that comes up when the honeymoon period with a test automation tool ends. I've seen teams whose intent/dream/goal was to create UI automation tests via a record and playback tool. They envisioned doing a single recording and all their tests would then be able to run at the push of a button.

It's a nice dream, but in order to really get the most out of your automation efforts, you really need to know what you're doing. It's impossible to automate all your tests, and quite frankly, you probably don't need to. I once worked for a manager who insisted that all bugs have an automated regression test for them. Think about that for a minute. All bugs. Every one of them. Even the ones that were citing spelling mistakes.

Needless to say, that wasn't a good use of my time, and I'm pretty sure my blood pressure was in the near fatal range for a while there.

When you're thinking about automation, think about what makes the most sense to automate. Smoke test? Yep. Get that going and integrate it with your build process. Regression test? Yep, but don't go crazy. Focus on the primary workflows of your application, don't try to automate a test for every bug you've ever logged. That whizbang new functionality that Sally just rolled into the build? Probably not.

That whizbang new functionality probably should not be automated yet because it's UI will change, and most likely will change a lot over the course of the release cycle. Instead, use automation to augment manual and exploratory testing for this feature. Let's say that Sally's piece is a report generator that pulls data from a database. Build some automation that sets up different data in the db, allowing you to test more types of her reports more quickly. Maybe you can set up a batch file that calls out to a diff tool and compares a report from today's build to a baseline.

The approach you take will largely determine your automation effort's success. Don't assume that just because you bought a record-and-playback tool that you've got test automation covered. The test tool is a dumb computer program; you're the one with the brain. Know when to employ the tools you have, when to use something else, and your testing will go much smoother.