Simple Tests

There's a great episode of Star Trek: The Next Generation where the omnipotent Q is made human. One of the tasks facing the crew during this episode is a moon that's losing orbit and threatening to crash into its planet. Q tries to help Geordi and Data prevent this disaster, and tells them that there's a simple solution. Geordi, excited, and asks what it is. Q's response: "Change the gravitational constant of the universe." Geordi's less than thrilled with this answer to say the least, but that's how Q would have handled the situation.

I see something similar when people are first evaluating automated test tools. They come up with a "simple test" and if the tool fails to perform that test, they immediately write the tool off as useless.

It's true there are times when a given automated tool isn't a good fit for a particular application, but keep perspective here. You, the tester, are on the same level as an empowered Q. You see everything in your application, and know how to make it work. The test tool doesn't have your ability to reason, to identify the cause of problems or to adapt. The test tool doesn't "see" the application the same way you do - it sees your application the way a *computer* sees it. You see a button labeled OK. Your test tool sees an extended Winforms control with a dynamically generated ID like cmdOK87823, or worse, the code may be obfuscated so that a test tool can't read any information from it.

The tool is operating on a much lower level playing field that you are, and it finding that dynamically generated button on an obfuscated application is just as impossible as it would be for Geordi to change the gravitational constant of the universe. So what to do in situations like this? You find ways to make the tool work with your application. In the TNG episode I mentioned, Data & Geordi found a way to send a warp field around the moon that would do something very similar to Q's solution. To find your dynamically generated controls, you could insert wildcards into your test scripts, so the tool would find cmdOK*, and thus match whatever identifier had been dynamically generated. You may need to run your tests against unobfuscated code. Just be aware that tasks like this may be a necessity, and you'll be much better off when it comes time to automate.