Keep Your Automation Visible

You and your automation are only useful if people know what you're doing. It's always a good idea to make sure the tools and scripts you've created have as much visibility as any other testing activity going on. Otherwise, someday someone's going to say "Hey, what does [insertYourNameHere] do all day, anyway?"

Here are a few thoughts on how you can keep people from asking that question:

Demonstrate your tools. Show the test & dev teams what you're working on. Do it during a lunch & learn and let them see what you've done. People may suggest some ideas on how to improve what you've created. Others may realize they have a use for what you've created. Either way, you're keeping your team informed on what you've built and how it can benefit them.

Send out updates. Between demonstrations, jot off a quick note to your team when you've updated a tool or script and how it can benefit them. When they realize you've added the ability to test foreign languages, or you've made it easier to verify values have been written to the registry, they'll be excited and will put those enhancements to work right away.

Blow your own horn. I hate doing this, but sometimes it's necessary. Your management needs to know that what you're doing is worthwhile, so sometimes you have to send off a mail saying "Hey, that tool I wrote helped Carla find 10 bugs in an hour" or "The smoke test I wrote caught a bunch of problems before it went to the test team, which kept them from wasting their time on a broken build." Especially in these economic times, being invisible is not the way to go.

Solicit feedback. Communication is a two way street. Always ask your test team "How could this be better?" when they see you implemented requests they asked for, they'll be more vested in your tools, and more excited about using them.

What other methods have you used to increase automation's visibility?

Your Automated Tests Are Stupid

Yes, they are. And not because you didn't put a lot of effort into planning them, nor are they stupid because you implemented them poorly. Your tests are stupid because the computer is stupid. It' s going to check exactly what you told it to, and only what you told it to. Keep that in mind; if you write a test that verifies a field in your app correctly displays the sum of two numbers, then your test will make sure that the sum of 2+2 = 4. That's great, but the test will miss a lot of things that a manual tester would catch. For example, if the results are suddenly displayed in bright pink; if they're in a font size too small to see; if they're displayed offscreen. Your automated tests will also miss any spelling mistakes in fields adjacent to the field being tested, which a manual tester would (hopefully) notice.

The point I'm trying to make here is that automated tests will not and should not replace your manual testers. Automation should be used to augment your manual testers; in the case above, you could have an automated test verify hundreds of thousands of various input combinations, and then have a manual tester run through once to do a spot check. That way you get the best of both worlds - you're assured that the app is functioning properly from both a technical and a human perspective.

Reflecting on a Demo

Last Friday I gave a TestComplete webinar to 314 people. The topic was Creating Script Extensions, which lets people extend the standard capabilities of TestComplete. I think it went really well overall, and I wanted to share a few of the things I learned/found helpful.

Let the audience know where you are in your presentation. I had a really short slide deck; most of the slides were used to track where I was in my agenda. This served as a transition between topics, and helped people keep track of where we were in the overall presentation.

Presenter Mode is a wonderful thing. PowerPoint 2007 has this great feature called Presenter Mode. This lets you have your presentation shown on one monitor, and your slides along with speaker notes and a timer on another. I was able to refer to my notes for each slide without having to print them off. This helped keep the presentation smooth, since I didn't have to go back and forth between a printout and what was displayed on screen.

Always have a cheat sheet. A good chunk of the demonstration involved teaching people how to write the code needed for a script extension. I had a blank Notepad++ window on the monitor I was presenting on, and on my other monitor I had the complete code I was writing, along with comments to help explain the code. This was insurance against me having a complete brain cramp while I was coding.

Load your applications ahead of time. No one wants to watch splash screens load or watch someone stumble through explorer windows and shortcut clicks. Have everything loaded in advance, and the demo goes smoother.

Order your applications in the taskbar. I used several programs during the course of the demonstration; PowerPoint, TestComplete, Notepad++, Paint.NET, Windows Explorer, and a couple of Notepad windows for my cheatsheets. It helps keep things smooth when the items in the Taskbar are ordered according to the order in which they'll be used. That way, you aren't searching for anything. There's a great little utility called TaskBar Shuffle that lets you reorder items in the Taskbar, in case you don't open apps in the order in which they'll be needed.

Poll your audience often. I was giving this demo remotely, so I couldn't see my audience. That meant I had no idea if they were engaged or not. I made it a point to ask them questions as I was presenting. Most of these were simple yes or no questions, but it was enough to keep them involved.

Have a wingman (or wingwoman). I had a co-worker riding shotgun with me during the demo. It was his job to answer questions about the telephone number needed to call in, and to alert me to any audio or visual problems that people were experiencing. This freed me up from having to respond to certain questions during the demonstration, and provided a mechanism to head trouble off before it became serious.

Breathe. I was really excited about giving this session. My energy level was pretty high, and I felt a little out of control in the first few minutes. I made myself slow down and take a breath, and was able to stay on a good pace for the session without rushing through things.

Record yourself. My wingman also recorded the demonstration. We wanted to be able to send this demo to clients who couldn't make it to the live session, but it's also a great tool for self-improvement. For example, I thought I had a really good handle on not saying "um" while presenting, but in listening to the recording I hear myself saying it, especially when I'm answering questions toward the end of the session. I'm going to keep working on that.

If you'd like to take a look at the recording, you can see it here (it's about 90 mins long). I'd welcome any feedback you can give.

You Have to Start Somewhere

I was speaking with a client recently who asked a lot of great questions. He was the manager of a small QA team, and really had a good feel for automation. I addressed everything he asked about, and then he told me that he had one other member of his team who had just a few "quick questions."

After speaking with this person for a few minutes, it was blatantly obvious that not only did he have no clue about test automation, he didn't have a clue about testing in general. To top it off, his initial attitude toward me was rather rude. My first reaction was one of mixed shock and frustration, but then I did what William Ury suggests in his negotiation book "Getting Past No" - I went to the balcony.

Basically this concept means you distance yourself from your emotions and your natural reactions - in essence, you step back from the situation and look at it with different eyes. I tried to put myself in this person's shoes. He was being asked to do something that he wasn't qualified for, and he was probably frustrated about that. Given today's economy, he was probably also scared that if he couldn't grasp these concepts, he might be out of a job.

So I stepped back and listened to his concerns and what he was trying to achieve. Then I addressed each concern in turn, and gave him lots of opportunities to ask questions or get clarification. The end result was a successful session. He admitted he had a lot of learning to do, but he felt really good about his chances to succeed.

My takeaway here is that everyone has to start somewhere. After doing test automation for so long, it's easy to forget that there was once a time when I didn't know a thing about it. I've posted a note above my desk that says "You were an English Lit Major" to remind myself that I asked all those really basic questions once too, and that perspective is part of the key to making other people successful as they learn about testing and test automation.

The Automation Mantra

I love tools. I love writing them, I love learning how to make the computer do more work for me. But it's easy to get caught up in the excitement of development and lose sight of what automation's really about. That's why I've adapted a mission statement that an old boss of mine had. Whenever I find myself writing a new app, or looking for a place to apply something I've created, I take a step back and say to myself -

"My job is to provide the QA team with helpful, easy to use tools. The output of these tools should be simple to understand and share."

If what I'm doing doesn't meet the above criteria, then I know I'm doing something wrong. It's a simple sanity check, but it keeps me from building things that aren't worthwhile. There are a lot of people out there who have been soured on automation because their experiences were with tools that did not follow these guidelines.

So when you're building tools for others, make sure they are:
Helpful - don't build a solution in search of a problem. Make sure you're really filling a need.
Easy to Use - don't make it harder to use the tool than to run the test without it.
Easy to Understand Output - put human readable English in your output messages, and store those messages in a file format that's easy to read. I've found XML/HTML are usually the best bets; PDF isn't bad either.

What other guidelines do you follow with your automation mantra?

Be an Educator, not an Enabler

This weekend I set up a computer at a relative's house. I unboxed it, plugged it in, went through the Windows setup screen. I set the timezone, removed some of the preinstalled crap-ware and finally installed MS Office. The entire process only took about 35 mins, but when I went home that night, I was disappointed in myself; I had missed out on an opportunity to teach my family how to do something with a computer.

Let's be real here folks, how hard is it to set up a new PC? This one had 3 wires. The CPU power cord, the monitor power cord, and the cable that connected the monitor to the PC. The keyboard and mouse were wireless, and just needed batteries installed. Loading Office is just a matter of inserting a DVD and clicking "Next" a few times. I could have walked them through the process and then they'd have felt smarter and more confident with the technology. Instead, I've inadvertantly fostered the belief that working on computers is hard, and borders on black magic.

It's like that with test automation, too. People who don't have an understanding of what it does and how it works see it as some sort of dark art. It's up to us to educate them so that they learn the right way to do things, rather than us always doing it for them. Good teachers help people figure out the answers for themselves, rather than give the answers to them. I'm going to try and be more like that in the future.

What have you done to help educate your team/group on automation? Have you been an educator or an enabler?