Alchemy, Not Magic

I once had a conversation with a tester that went something like this -

"We need the automation to be able to recover from errors. I don't know what those errors will be, or what steps will need to be performed to address those errors, but the upshot is that the automation needs to get the AUT back into a good state before continuing."

I explained that in order to build this kind of logic, at least some knowledge was needed of what constituted an error, and how the error was to be addressed. They rather heatedly told me that if the automation couldn't "just know" that it was not going to be helpful.

Here's the thing. We need to set reasonable expectations for test automation. Automation can make testing easier, and can speed up tedious processes. However, automation is not some magical, omniscient force that can interpolate how your application works and how to best recover from problems. If it could, we wouldn't need testers or developers - all our software would write and test itself. Anyone who tells you that automated tests will just magically resolve themselves with no direction on your part (they might be cute and use the word "automagically" - watch out for those people) is someone who's selling something, or is clueless.

Automation is a tool. It's not magic. It's not smarter than you are. It's only as smart as you make it, and it can only do what you tell it. Your automation can never, and will never "just know" how to test your application. The computer is a stupid box. It's just fast. If you don't tell it what to do, it will do nothing, although it will do nothing very quickly.

In short, automation is not magic, it's more like alchemy. It can be complicated at times, but the end result of making the testing process smoother/easier/faster is akin to turning lead into gold.

The Julia Child Approach to Demos

When you're giving live demos, it's always handy to have everything you need for that demo ready to go. Have powerpoint already loaded, have the application you're demo-ing already loaded, and have anything else ready to go, too. If you'll need to show a calculation, have Calculator already running. If you'll need to write some code in Visual Studio, have that already running & open to a blank project. The idea here is that your audience should never have to watch a splash screen or "Loading" dialog. Before I run a demo, I launch all the apps I need, and organize them on the Windows task bar so that they're in the order that I'll use them. Then I just click right through the task bar as I go from one activity to the next.

Additionally, if you're going to demo any feature that takes longer than 10 seconds to complete, consider having a "before & after" of that feature, where you walk through the steps needed to perform the action (the before), and then have a finished version of the feature ready to show off (the after).

(A note here - always tell your audience about how much time the operation should take to complete. You don't want to misrepresent how fast your application works, that's a sure-fire way to lose credibility in your customers' eyes. I usually say something like "This operation usually takes about 5 minutes to complete, so instead of making you stare at my hourglass for that much time, I've already got a finished sample right here")

I call this the Julia Child approach, because I was first exposed to it while watching Julia Child's cooking show back when I was a kid. Julia would have all her ingredients and utensils ready, so she'd smoothly go from one prep task to the next. For example, if she needed to chop some tomatoes, she already had the tomatoes washed, and there was a cutting board and a knife right there. You didn't need to watch her hunt around for anything, she had it ready to go. Then she'd walk you through all the steps to make a cake, then put that cake in the oven. Then she'd move over to a second oven, and remove a cake that had finished cooking. It was a great way to show both the preparation and the finished product, without needing a lot of filler in the presentation.

So having all the pieces of your demo ready to go will make the presentation go smoother, and having a "before & after" prevents awkward lapses in the presentation. All in all, this approach helps ensure a clean, smooth presentation that your customers will eat up. Bon Appetit!

Automate in Straight Lines

Everyone's heard the adage "the shortest distance between two points is a straight line." This holds true in test automation as well. I recently spoke with someone who was struggling to get a UI automation tool to run against an application. This lady had no experience with test automation before, and the application was not "automation friendly." As such, she had spent almost an entire week struggling to get tests running.

I spoke with her on the phone for about 30 minutes, and during the course of the conversation, the intent of what she was doing came out. She was not testing the software, she was trying to populate a database with information, and this database would then be used for training purposes. She'd need to repopulate the db each time there was a class to teach. This GUI automation scenario she was trying to create would take hours to run, but she hoped she could start it on the night before a class and then have it be ready in the morning.

I asked her why she didn't just write a plain old SQL script that would populate the db, then create a backup of the database and restore it each time she needed to teach a class. There was silence on the other end of the line.

Look for the simple solutions. Databases can be written to directly, and then restored, over and over again. Webservices can be manipulated directly, methods can be called from dlls directly, the Windows Registry can be manipulated directly. If your intent is just to get data from point A to point B, do it in a straight line via script. Adding other layers of complexity (UIs, GUI tools, etc) is kind of like trying to go from New York City to Boston by way of Los Angeles.