QTP pitfalls for WinRunner users (Part II)

Today was another day of fiddling around with QTP. The VBScript language of QTP is still a bit new to me and I stumbled into some neatly disguised traps:

1. The .exist(n) method is in seconds
Normally I use the .exist() method for synchronization, but today I used it for checking if the right finalise page was reached or that I had to cope with an data validation error somewhere. I do not bother test objects, because at this moment I am not testing, but just doing an automated data entry project with some complex datamodels. (This is where the classes are hopping in I mentioned before).
To test my code, I started my QTP script and went for my lunch. When I got back, only one set of data was entered and the screen was synchronizing on a page. And synchronizing. And synchronizing.

After pressing the stop button and debugging my code, I found out that the wait time in exist(n) is expected in seconds, while the help file mentions milliseconds!
My script was waiting for a never appearing page for 10.000 seconds instead of 10.000 milliseconds.
Not very uncommon for the WinRunner user by the way (obj_exists() and win_exists() works almost the same), only misleading because of incorrect information the help file.

2. Location is optional in object identification
When there are multiple object in your AUT that match the same description in the object repository, QTP just picks the first one when the ordinal properties are set to 'none'.
And this property is always set to none whenever you learned the object while it was unique on the screen.
In WinRunner, you used to get the error message E_NOT_UNIQUE, but QTP is figuring it out all by itself, even with Smart Identification switched off. Something to keep in mind.

3. Eval() is not exactly eval();
Eval() in QTP is used for comparison, while Execute() is used for assignment:

a = 2
msgbox eval("a = 2") ' displays 'True'
msgbox a ' displays '2'
msgbox execute("a = 3") ' displays nothing, a is set to 3
msgbox a ' displays '3'

Keep this in mind if you used the eval() function in WinRunner for assignment of virtual variables. It won't work in QTP and you have to use execute().

4. There is no fast way to export the object repository to plain text
Unlike WinRunner, where the GUI map was just a plain text file, it is not possible to easily export (and manipulate) the object repository. It is possible though, but you'll have to do it through COM automation on the QTP application. I tried it today, but I failed miserably because I could not access the ActiveX object (probably not enough authorization on my workstation).
However, if I get it working, I will post the code on Automated Chaos.

Lowlevel Code Optimization in WinRunner

At some moment in your automated test project you enter the phase you are questioning yourself: "Weren't we automating the tests because it is fast? Why are my tests running so slow!?!". Then it is time to review your code and to optimize it a bit.

In this post, I want to discuss low level optimization. As WinRunner does not have a smart compiler (it is more an interpreted language then a compiled one), we have to do all optimizations by ourself.

1. Order of function calls in a condition
When you use and and or constructs in conditions ('if' statements for example), you have to think about the order:

Let's say, we have two checks, a time consuming (fncSlowCheck) and a fast one (fncFastCheck).
When you use an AND or an OR construct, you need to put the fastest compare first:

if (fncFastCheck() == E_OK && fncSlowCheck() == E_OK) { ... }

if (fncFastCheck() == E_OK || fncSlowCheck() == E_OK) { ... }

WinRunner will first evaluate the fast function. When this evaluates to false, the second function will not be called in case of the AND construction, because the total if statement can never become TRUE, and the if statement is stepped over.
With the OR, it is just the other way around. When the first function evaluates to TRUE, the second function will not be called.

This behavior is called lazy evaluation and is something you have to keep in mind when you use functions in conditional statements that can impact the application under test.

if (edit_set(myEdit1, "foo") != E_OK && edit_set(myEdit2, "bar") != E_OK) {
    write2report("Something went wrong setting myEdit1 and/or myEdit2", ERROR)
}

Besides this is a crappy way of reporting, the second edit will never be set as soon the first edit_set() evaluates to an error.

2. Use switch / case constructs. They are fast!
The reason why switch / case are fast is because jmp (jump) commands in assembly are faster than cmp (compare) commands used with each loop iteration. Combined with the so called "fall through" mechanism, they cannot be beaten by loops. For a detailed article, see Duff's device on wikipedia.

3. ++i is faster than i++
Make it a good habit to use ++{variable} in your for constructs:

for (i = 0 ; i < 100 ; ++i)

I absolutely noticed the difference since I had some large multi dimensional arrays I had to iterate through.
One note: Keep in mind the order of handling of the incrementation. Do not blindly search and replace all {variable}++ with ++{variable}, this will mess up your test.

4. Calls to external functions can be slow
When you use external functions in time critical processes, it is a good habit to check the performance of these functions. Once, I had external and() and or() functions (they are not provided by WinRunner) and I used them to make a bit collection of matches of a table row. With each table row check, the correct bit was set to 1 or 0 in case of a match or a non-match.
But this rowcheck function was very slow. More then 3 minutes for a 25x25 table for example.

First, I didn't bother about the bad performance, we did a lot within a check: Negative checks, regular expressions and other exotic stuff to support the testers. But the rowcheck function got used more and more and the long idle times became annoying.
We created a lookup table for the powers of 2, optimized the loops and conditional statements, but it still underperformed.

Then, we measured the time for a "x = and(a, b);" call and it was 1 tenth of a second. This means more then one minute in case of 25x25 checks, and we not only used it onced, but three times in one iteration.
I assumed that the and() function had to be fast, because it was fast in C, the language the external lib was written in.
After changing the bit collection through and()s and or()s to an array and performing calculations on the array (the product as an and(), the sum for an or()) the performance was increased to 23 seconds for a 25x25 table check. Even with regular expression and negative testing in place.

5. Function calls can be slow
Whenever a function is called, a function is created on the stack, initialized, executed and destroyed. Processors are fast nowadays, but it still takes some time.
I used to use isEmpty({variable}) and isNotEmpty({variable}) for a check on empty or not. The only thing the function did was:

public function isEmpty(inValue) {
    return (inValue == "");
}

It seemed smart on that moment, because if we created other definitions for empty, we just have to enter it in the function to implement it everywhere in the system. Unfortunately, we never came up with other definitions. We noticed, the isEmpty(myVariable) function was seven times as slow then a normal myVariable == EMPTY statement (EMPTY equals "" in our system).
With this knowledge we replaced all isEmpty() functions in time critical functionality, such as the table row search function mentioned above.

A little side note:
    myVariable == EMPTY
or
    myVariable == ""
does not make any difference in WinRunner.

Last word
As mentioned before, this is how you optimize code on a very low level. As long as your design is not right, use senseless synchronization timers or bad synchronization mechanisms, the performance of your test will not increase significantly. Optimization must be in balance on all levels of the test process; From scripting automated tests to requirements and risks.

QTP and Tricks with Classes

As a hardcore WinRunner test automater, I still not like the "one line, one statement" approach of VBScript, but the one thing I like is working with classes. I'm still playing around with it a little bit and I learn a bit each day. The thing I did today isn't new I think, but for me it was.

A little about my situation. We are (ab)using QTP as a dataentry tool. We can only use the frontend, because the backend is protected. (so long for third party software). There will be a backend interface soon, but soon in the ICT is still counted in multiple months and a quicker solution had to be found. Maybe that is why QTP is chosen. On the contrary, I think the program manager felt for the 'Q'. Right now, the data is coming from multiple sources and will be directed to the frontend with QTP.

And this is where I discovered a neat application of classes for the use in dataentry. I created a class, let's say "customer" and I redirected the class into a function as a method of that class.
Time for an example:

First, my class 'customer':

class customer
    public name
    public address
    public phone
end class

and my function to enter the customer into the frontend:

public function enterCustomer(byRef myCus)    ' Use byRef for speed and to save memory

    call navigateToCustomerCreate()        'I like functions more then subs

    with Browser(zzz).Page(yyy)
        .WebEdit("Name").Set myCus.name
        .WebEdit("Address").Set myCus.address
        .WebEdit("Phone").Set myCus.phone
    end with

end function

now, I added a function into the customer class to enter the customer:

class customer
    public name
    public address
    public phone

    public function enterData()
        call enterCustomer(me)        'The 'me' is referring to it's own class object 'customer'
    end function
end class

This creates a method for my customer object to enter my data.

Because a customer can have multiple accounts, I created a class account and that one is added as a dynamic array in the customer class. And for data integrity I created a validate method that checks if all mandatory fields are filled and if all data is in the correct formatting (hurray for regular expressions).
But I can add as much functionality as I like in a ordered way, the only call I have to make to get it all working is using the enterData() method in my driver script for entering all my data into the frontend.

The main reason why I use this method, is that I only have to write an import function for all three types of import (plain text, xml and through an ODBC connection) and redirect the data into the class. At the end I call myCustomer.enterData() and it is done.

In the mean time, testing my functions is very easy, I just create an object of the correct class and test the function by calling enterCustomer(myCustomer). I don't have to bother validation rules because they are only executed when the enterData() method is called. And because the method validate() is a public method in the customer class, I do not have to enter the data when I want to test the validation of it.

It seems a lot of work for only adding a customer but in the real situation, the datamodel contains a lot more objects with a lot more fields. This way, it keeps my code clean, structured and also important: testable.

WinRunner and why I like TSL

I like TSL. Not because of its lack of classes, structs, function pointers or other fancy eighties C stuff, but because you can write powerful code on a single line. I like it even more since I am now developing my test automation in QTP. Yeah, that's right, the tool that uses crippled Visual Basic Script.

One nice thing in WinRunner is that you can use assignments inside functions or validations like:
for(i = 0; (rc = obj_exists(thisButton = “{class: push_button, location:”i”}”)) == E_OK; ++i)
    button_press(thisButton);
pause(i-- “ button” (i != 1 ? “s were” : “ was”) “ pressed.”);

The three line above contains interesting code, I'll walk through it.

First, this contains an assignment in function:
obj_exists(thisButton = “{class: push_button, location:”i”}”);

And second, an assignment on validation like:
(rc = obj_exists(myObj)) == E_OK
thisButton = “{class: push_button, location:”i”}” is replaced by myObj to make it more readable.

Notice the parenthesis around the assignment; Normally assignments have a lower precedence, that is why we need it.
(Explanation:
If you use code like this:
rc = obj_exists(myObj) == E_OK
first obj_exists(myObj)) == E_OK is evaluated to TRUE or FALSE. Since rc gets that value assigned, rc becomes FALSE in case of a none existence of the object and TRUE otherwise, making the statement as buggy as hell; FALSE and E_OK both have the same value: 0, causing rc getting a E_OK value in case of a not existing myObj.)

The last line contains a short cut for an if else statement:
(validation ? value if true : value if false)
This is great when you want to write fast code in a (log)message. Otherwise the same line would take you at least four lines if you want to state it properly.
1. if (i == 1)
2.    pause("1 button was pressed.”);
3. else
4.    pause(i "buttons were pressed.");

The code in the example above is nothing worth in the sense of readability, but more to show the power of writing C-like lines of code. I would challenge the same functionality in a Visual Basic script like language. I think it will be 5 times more lines of code and a lot of debugging, since it is easily forgotten that VB(S) functions in "for" statements are evaluated only once. More on this another time.

WinRunner Hidden Coding Features

WinRunner has certain features that are not commonly known. This is a list with the most powerfull ones:

print {statement};
In non-batch mode, this will print the outcome of a statement/variable or string to an output screen. Great for debugging and much faster (and less annoying) then the pause() function. This function is not documented in the help files, only as a registered word (a word you can not use as a name for a custom variable or function).

sprintf(format, variable, ...);
This function works like the (s)printf in C, but without an out variable. The return value is the formatted string.

Example:
percCompl = 78.22845;
A = sprintf(“Completed: %0.2f%%”, percCompl);
# A == “Completed: 78.23%”
For a complete overview of format characters, search the internet on sprintf() or get a C/C++ guide.

Although this function can be found in the WinRunner users guide, it is highly underestimated. It works great for getting "check" variables in the correct formatting without bothering string manipulation.

Use Add Watch as direct execution window
Use the Add Watch to see variable values during runtime and/or after pausing a test.
While pausing a test, you can enter direct functions into the add watch window and evaluate the return value. So you can check during runtime if objects are existing and if certain functions are working properly. The shortcut to Add Watch is CTRL+w.

Use the call chain in the Debug menu function to find out function callers and callies.
Within the callchain, you can jump from and to functions that are on the call chain; the values of the variables are still in place and you can use add watch to evaluate them (see section above).

You can also use the call_chain_get_depth() and call_chain_get_attribute() WinRunner functions to write extensive debugging information to a log file. When you are used to the try/catch mechanism, you will certainly appreciate these two functions.