Selenium onLoad() handling

At APT, we’ve recently completed a transition to using Selenium’s Java RC¹ for our regression testing framework. One of the excellent things about Selenium RC is the ability to add functionality with relative ease.

In this entry, I want to briefly cover how we get around the onLoad() event caveat of Selenium. Since Selenium works by Javascript injection, it is unable to catch onLoad() events, which can prove to be troublesome when dialog boxes are hit onLoad(). Our code base is relatively large for such a small company and in testing we are really pushing the limits of Selenium. Our goal is to have all of our front-end testing done through Selenium, which means that it needs to elegantly handle Javascript errors and onLoad confirmations/alerts.

When executing an action such as click(), Selenium can hang if a Javascript error is hit. In order to handle this, we now spawn a separate thread to handle any action that might alter the state of the page. The main thread is then able to poll and check if the action thread has hung and if it has we call an AutoIt script² which closes the dialog box and logs an error to our database. This is especially powerful because it lets our tests continue to run with virtually no human interaction.

By leveraging anonymous classes in Java, any Selenium function can quickly be overloaded and run in a custom multithreaded execution thread, as well as make any encountered errors available for the main thread to handle.

This is just an example of how we are automating Selenium, which is crucial in order for it to work with our distributed framework. In my next blog entry I would like to cover how we are accomplishing quick distributed compilation.

- Dan Hackner

1 – http://seleniumhq.org/projects/remote-control/

2 – a Window’s GUI level scripting language. http://www.autoitscript.com/autoit3/

VBScript Syntax Checking

This post describes how to use VBScript to check the syntax of a given block of VBScript code.

The Motivation:

At APT, one software suite we use to run automated GUI tests of our web-based software is Mercury QuickTest Professional. QuickTest runs tests written in VBScript that click through a prescribed path in our software to make sure we don't throw any errors.

Rather than store hundreds of static .vbs files containing the definition of tests, we store our test definitions in a SQL database and dynamically generate the corresponding .vbs files on demand. We do this for a few reasons:

1. It makes test maintenance easier (you maintain only the structure of the test, not the code's syntax)
2. It enables users who are unfamiliar with VBS to author and maintain tests
3. By introducing separation between the test's definition and execution, we can change how we execute our test without changing our test's definition (for example, migrating test execution to another platform such as Selenium, which can execute different languages)

The Problem:

One of the challenges of dynamically generating code is making sure that what you generate is syntactically valid. Making the assumption that your generated code will be perfect is generally a bad idea, because:

• Your code-generation code may contain errors
• As you modify your test definition model, the code-generation code can fall out of sync, resulting in invalid generated code
• You'll likely want to support user-entered "custom" sections of code, which you can't assume will be syntactically valid

The Solution:

After we generate the VBS code for the requested test, we check its syntax prior to executing it by calling the function defined below:

Public Function checkVBSSyntax(vbsCode)
    ' Save any pre-existing error so we can revert to it
    oldErr = Err
    Err.Number = 0

    On Error Resume Next
    ' Execute the vbs code to see if it throws an error
    ExecuteGlobal vbsCode
    hasError = Err.Number <> 0

    On Error Goto 0

    ' Revert to the pre-existing error
    Err = oldErr

    checkVBSSyntax = (Not hasError)
End Function

What we're doing here is simply executing the questionable code in the global scope to see if it throws an error. If it throws an error, we assert that it must be due to a syntax error.

A couple of notes:

• This function assumes that the questionable code contains only Function and Sub definitions; that is, no code exists outside of a Function or Sub declaration. If it does contain code outside of a Function or Sub declaration, that code will actually be executed (not just checked for its syntax). If you have such code, wrap it in a Public Sub main() ... End Sub statement.
  


• This has the side effect of defining all the Functions / Subs in your questionable code in global scope; that is, after calling this function, you'll be able to call any functions defined in your questionable code, as if you included it as a function library. As a result, you'll want to be careful that Function / Sub names in the questionable code don't collide with Function / Sub names in your syntax checking code.