Getting Started with Automation

Every so often I’ll see posts on message boards like this: I need to automate tests, what tool should I use? 

And unfortunately, that’s the wrong angle to approach the problem from. The first question you need to ask yourself is “what programming language is the app I’m testing developed in?” That seems like a very basic question, and some people may be insulted by it, but I have worked with teams who didn’t know the answer. Keep in mind that there may be more than one language in use – an app developed in ASP.NET may also have a SQL server back end, so you’d need to know C# and SQL. Once you know what language your app is written in, the next question is, “Can I code in those languages?”

If not, start by learning about that language. I’m not saying that you need to become a full blown .NET or Java developer, but you should understand the language, how it works, and how to write some basic code. Now, this is usually where an oldschool QA manager would have a stroke. “My people aren’t developers,” he says. “I just want them to click a record button and then have everything work.”

That’s a lovely idea, but it’s not reality. The cold truth of the matter, folks, is that when you automate, you are writing code that tests other code. That’s exactly what record and replay tools are doing, but they abstract it up a level or two so that users don’t see that code. Problem is, those tools can only be so smart. Eventually, you’re going to encounter something in your application that a record & replay tool can’t handle, and then you’re stuck. The only way to become unstuck is to code your way around the problem.

So start out by learning a bit about the language your app is written in. This will help you learn how your app works under the hood, which then means you can start tinkering with different ways to automate things. Back before I knew SQL, I had to modify all my test data via my app’s UI. That was a laborious, time consuming process. Then I learned how to write Update statements, and that let me modify the data much faster and more efficiently than before. I’ve seen this many times, where the person who does the most effective automation is the one who understands how to code. 

It’s a worthwhile endeavor that will not only improve your automation efforts, but will increase your value as an employee as well. 

Helpful Tool: PasteCopy

Often when I'm giving demos, I'll need to type in a sentence or two into a text field. This is no big deal, I can type pretty fast, but we all have those days when we can't type to save our lives. And invariably, this happens when lots of people are watching. To help avoid this, I've been using a freeware clipboard manager called PasteCopy. This lets me store off sentences that I will use in demos, URLs to send to participants who are in webinars, or paths to files that I commonly need. I keep PasteCopy open on another monitor, and when I need to get at one of the stored strings, I just pop over to that monitor, click on the desired string in PasteCopy, and then it's ready to be pasted into the desired field. It works beautifully and has saved me lord knows how many incorrect keystrokes.

PasteCopy installs in about 3 seconds and has a very light footprint. You can check it out at http://www.pastecopy.net/

Alchemy, Not Magic

I once had a conversation with a tester that went something like this -

"We need the automation to be able to recover from errors. I don't know what those errors will be, or what steps will need to be performed to address those errors, but the upshot is that the automation needs to get the AUT back into a good state before continuing."

I explained that in order to build this kind of logic, at least some knowledge was needed of what constituted an error, and how the error was to be addressed. They rather heatedly told me that if the automation couldn't "just know" that it was not going to be helpful.

Here's the thing. We need to set reasonable expectations for test automation. Automation can make testing easier, and can speed up tedious processes. However, automation is not some magical, omniscient force that can interpolate how your application works and how to best recover from problems. If it could, we wouldn't need testers or developers - all our software would write and test itself. Anyone who tells you that automated tests will just magically resolve themselves with no direction on your part (they might be cute and use the word "automagically" - watch out for those people) is someone who's selling something, or is clueless.

Automation is a tool. It's not magic. It's not smarter than you are. It's only as smart as you make it, and it can only do what you tell it. Your automation can never, and will never "just know" how to test your application. The computer is a stupid box. It's just fast. If you don't tell it what to do, it will do nothing, although it will do nothing very quickly.

In short, automation is not magic, it's more like alchemy. It can be complicated at times, but the end result of making the testing process smoother/easier/faster is akin to turning lead into gold.

The Julia Child Approach to Demos

When you're giving live demos, it's always handy to have everything you need for that demo ready to go. Have powerpoint already loaded, have the application you're demo-ing already loaded, and have anything else ready to go, too. If you'll need to show a calculation, have Calculator already running. If you'll need to write some code in Visual Studio, have that already running & open to a blank project. The idea here is that your audience should never have to watch a splash screen or "Loading" dialog. Before I run a demo, I launch all the apps I need, and organize them on the Windows task bar so that they're in the order that I'll use them. Then I just click right through the task bar as I go from one activity to the next.

Additionally, if you're going to demo any feature that takes longer than 10 seconds to complete, consider having a "before & after" of that feature, where you walk through the steps needed to perform the action (the before), and then have a finished version of the feature ready to show off (the after).

(A note here - always tell your audience about how much time the operation should take to complete. You don't want to misrepresent how fast your application works, that's a sure-fire way to lose credibility in your customers' eyes. I usually say something like "This operation usually takes about 5 minutes to complete, so instead of making you stare at my hourglass for that much time, I've already got a finished sample right here")

I call this the Julia Child approach, because I was first exposed to it while watching Julia Child's cooking show back when I was a kid. Julia would have all her ingredients and utensils ready, so she'd smoothly go from one prep task to the next. For example, if she needed to chop some tomatoes, she already had the tomatoes washed, and there was a cutting board and a knife right there. You didn't need to watch her hunt around for anything, she had it ready to go. Then she'd walk you through all the steps to make a cake, then put that cake in the oven. Then she'd move over to a second oven, and remove a cake that had finished cooking. It was a great way to show both the preparation and the finished product, without needing a lot of filler in the presentation.

So having all the pieces of your demo ready to go will make the presentation go smoother, and having a "before & after" prevents awkward lapses in the presentation. All in all, this approach helps ensure a clean, smooth presentation that your customers will eat up. Bon Appetit!

Automate in Straight Lines

Everyone's heard the adage "the shortest distance between two points is a straight line." This holds true in test automation as well. I recently spoke with someone who was struggling to get a UI automation tool to run against an application. This lady had no experience with test automation before, and the application was not "automation friendly." As such, she had spent almost an entire week struggling to get tests running.

I spoke with her on the phone for about 30 minutes, and during the course of the conversation, the intent of what she was doing came out. She was not testing the software, she was trying to populate a database with information, and this database would then be used for training purposes. She'd need to repopulate the db each time there was a class to teach. This GUI automation scenario she was trying to create would take hours to run, but she hoped she could start it on the night before a class and then have it be ready in the morning.

I asked her why she didn't just write a plain old SQL script that would populate the db, then create a backup of the database and restore it each time she needed to teach a class. There was silence on the other end of the line.

Look for the simple solutions. Databases can be written to directly, and then restored, over and over again. Webservices can be manipulated directly, methods can be called from dlls directly, the Windows Registry can be manipulated directly. If your intent is just to get data from point A to point B, do it in a straight line via script. Adding other layers of complexity (UIs, GUI tools, etc) is kind of like trying to go from New York City to Boston by way of Los Angeles.

MS Office Automation

I've worked with a lot of people who have wanted to automate MS Office apps. Sometimes they're trying to just write results data out to a Word doc, other times, their company builds a plug-in to an Office app and they're trying to create automated tests for that plug-in. And they all start out by pointing a commercial record & playback tool at the app and clicking that red "record" button. At best, this results in tests that click on obscurely named objects, or at worst, tests that are driven exclusively by X/Y coordinates.

You see, MS Office apps (Word, Powerpoint Excel, Outlook) are not record & playback friendly. I learned this the hard way a long time ago, and I'm hoping to spare others the same pain. You will not be able to create a reliable, robust set of MS Office tests via a record and playback tool. The only way I've found to effectively work with Office apps is via scripting. The Office object model lets you programmatically access any bit of text, cell or slide. For example, this code launches Word, opens a new document, and writes Hello:

Set objWord = CreateObject("Word.Application")
objWord.Visible = True
Set objDoc = objWord.Documents.Add()
objDoc.Selection.TypeText("Hello")


If you try to do those same actions via record & replay, your results will be spotty at best. So when you need to work with Office apps, automate them via their object model. A little known fact is that MS ships help files with office that describe each app's object model, along with coding examples. The location of these files is [OfficeInstallDirectory]\[OfficeVersion]\1033
The app's help files are:

• Word - VBAWD10.chm
• Excel -VBAXL10.chm
• Powerpoint - VBAPP10.chm

There's also an online version here: http://msdn.microsoft.com/en-us/library/y1xatbkd%28VS.80%29.aspx

Keep these handy next time you're doing MS Office automation, and your automation efforts will be much more successful.

Silver Bullets and Snake Oil

After doing test automation for most of my professional life, I'm really tired of vendors claiming that automation is a silver bullet. I'm also tired of test managers claiming that automation is snake oil. Here's the thing. The silver bullet pitch ("Automate all tests with the click of a button; no coding knowledge or original thought needed") is snake oil. I've said before on this blog that if you want to automate, you need to learn how to write script. I still hold by that. So it's the vendors' claims that are the problem, not automation itself.

Let's look at this using a different example - the microwave oven. The little cookbook that came with my microwave claims the microwave can cook any food just as good as the traditional methods. But that's really not true. While the microwave can cook just about everything, the food comes out different. Chicken comes out rubbery, for example. The button marked Popcorn cooks popcorn for too long, and often burns it. There's a recipe for cooking a small turkey, but I'm not courageous enough to try that.

Now, the cookbook is making a silver bullet pitch. "The microwave cooks everything just as good as traditional means" And, based on my experience, that's not the case. However, does that mean that I should throw the microwave away, and denounce all microwave ovens as worthless? No. It means I still use the regular oven to cook chicken, and I manually key in how long to cook the popcorn. In short, I adapt the microwave to my needs and use it for what it does well. I still have a grill, a deep fryer, an oven and a stovetop. I microwave what's appropriate for me to microwave and that's it.

Same thing for automation. I would never recommend (or even try) to automate 100% of your tests. I would recommend trying to automate tasks that are difficult or impossible to perform by hand. I would recommend automating "prep" tasks, like loading a database, building strings for use in boundary testing. I would recommend using tools to automatically parse log files for errors, rather than trying to read them by hand. The application of automation is the important thing here; you need to be smart about what makes sense to automate, just like you need to be smart about what you try to cook in the microwave.

Silver bullets are for werewolves. Snake oil is for 19th century hucksters. Automation is neither. Automation is for testers and developers who want to put in some effort to speed up their existing processes. It's just a tool. That's all.