June 6, 2008

QTP and the curse of the interview questions

Quick Test Professional is getting popular. It wins easily from good-old-and-soon-to-be-disposed WinRunner. Because QTP is relatively new, all over the internet the knowledge bases are still building up. That is good thing. Not a good thing in my opinion is the way people are trying to get a job. Just do a search on “QTP interview question” or better “QTP interview answers” and see for yourself; there seems to be a large market for publishing answers on interview questions. And this is something that scares me.

When a project wants to start automating their tests, it is automatically a shift in the test process. Testers have to be stricter in defining and formatting their data, business analysts do have to make clear requirements and the development team has to cope with standards. None of them is a bad thing, unless you don’t want to improve your test and development process of course.

Most of the time there is no automated test available yet and the project is searching for a specialist who can design and implement the automated test. When the decision for test automation is made, the business puts lots of assets on the automated tests that have to be developed. Besides the significant costs, the assets also include the change and implementation of the architecture, test framework, risk model, the fitting on the manual tests and the test process.

And then, the interviews. The interviews with potential test automation specialists are most of the time performed by test managers, project leads and colleague testers, none of them very experienced in test automation. The most technical guy of them probably had played with the test automation software a bit, and he is the likely one to ask some technical questions.

But how can you make a difference between candidate A and candidate B if the answers to those questions can be learned by head? Or even worse, what if you only have candidates that have their knowledge from the internet? You pick the best one and you are satisfied with that, but it is like choosing between a blind carpenter and one without hands.

Right now, there are more people who like to work with WinRunner than to actually learn it. Soon enough, QTP will end up with the same statistics. That is not a good sign for the automated test business in general. Inexperienced people on projects claiming the contrary might result in failures. Project managers don’t like failures and when test automation gets associated with failure, you'll get a hard time convincing them of the opposite.

Certification is a hot topic right now. I am not a big proponent of pointlessly requiring a certificate for every tool you want to use. On the other site, it sifts the wheat from the chaff at forehand and it gives an instrument to projects, inexperienced with test automation, letting them easily select the experts from an applicant pool.

1 comment:

Anonymous said...

I've been puzzled by those QTP interview question sites for some time. You don't see sites like that for C#, AutoCAD or other professional tools. What is it about QTP that causes this?

As for screening out candidates who only know a bit about QTP from reading these sites, that's what a CV is for. A company's first automated tester should be somebody with a strong resume and verifiable references; not somebody who hasn't ever had a job with QTP before. Once test automation is established, then you can take a risk on less experienced testers, and that first tester -- the one with years of experience -- should be the one asking technical questions in the interview.