VBScript Syntax Checking

This post describes how to use VBScript to check the syntax of a given block of VBScript code.

The Motivation:

At APT, one software suite we use to run automated GUI tests of our web-based software is Mercury QuickTest Professional. QuickTest runs tests written in VBScript that click through a prescribed path in our software to make sure we don't throw any errors.

Rather than store hundreds of static .vbs files containing the definition of tests, we store our test definitions in a SQL database and dynamically generate the corresponding .vbs files on demand. We do this for a few reasons:

1. It makes test maintenance easier (you maintain only the structure of the test, not the code's syntax)
2. It enables users who are unfamiliar with VBS to author and maintain tests
3. By introducing separation between the test's definition and execution, we can change how we execute our test without changing our test's definition (for example, migrating test execution to another platform such as Selenium, which can execute different languages)

The Problem:

One of the challenges of dynamically generating code is making sure that what you generate is syntactically valid. Making the assumption that your generated code will be perfect is generally a bad idea, because:

• Your code-generation code may contain errors
• As you modify your test definition model, the code-generation code can fall out of sync, resulting in invalid generated code
• You'll likely want to support user-entered "custom" sections of code, which you can't assume will be syntactically valid

The Solution:

After we generate the VBS code for the requested test, we check its syntax prior to executing it by calling the function defined below:

Public Function checkVBSSyntax(vbsCode)
    ' Save any pre-existing error so we can revert to it
    oldErr = Err
    Err.Number = 0

    On Error Resume Next
    ' Execute the vbs code to see if it throws an error
    ExecuteGlobal vbsCode
    hasError = Err.Number <> 0

    On Error Goto 0

    ' Revert to the pre-existing error
    Err = oldErr

    checkVBSSyntax = (Not hasError)
End Function

What we're doing here is simply executing the questionable code in the global scope to see if it throws an error. If it throws an error, we assert that it must be due to a syntax error.

A couple of notes:

• This function assumes that the questionable code contains only Function and Sub definitions; that is, no code exists outside of a Function or Sub declaration. If it does contain code outside of a Function or Sub declaration, that code will actually be executed (not just checked for its syntax). If you have such code, wrap it in a Public Sub main() ... End Sub statement.
  


• This has the side effect of defining all the Functions / Subs in your questionable code in global scope; that is, after calling this function, you'll be able to call any functions defined in your questionable code, as if you included it as a function library. As a result, you'll want to be careful that Function / Sub names in the questionable code don't collide with Function / Sub names in your syntax checking code.
  

Fault-Tolerant ColdFusion Error Reporting

ColdFusion’s default reporting mechanism for uncaught errors is to dump the error information to the console. This has several disadvantages, two of them being that the console would have to be monitored and whoever is monitoring it is limited to the dumped information. To address this issue, ColdFusion Administrator has a setting for a Site-wide Error Handler. This is a custom template to execute when an error is encountered, and allows for sophisticated post-processing of the error.

At APT, we take advantage of this feature and have a very robust Site-Wide Error Handler. Among other things, it retrieves additional information about the user and setup, retrieves information about the product’s state, diagnoses the error, logs the error to a database, and sends an e-mail to relevant engineering and delivery team members. But there is a catch: if an error is encountered within the Site-wide Error Handler it stops processing and dumps that error to the console. Should this happen, we are right back where we started.

The solution is simple: make sure your Site-wide Error Handler never fails. However, if you have a complicated template, this is easier said than done. In this situation, we must make the Site-wide Error Handler fault-tolerant.

How do we achieve fault-tolerance? The foundation is liberal use of try-catch blocks within your Site-wide Error Handler. While it may at first appear sloppy and clutter up the code, it is critical for isolating the effects of errors, making sure the maximum amount of processing takes place, and ensuring the most information possible makes it back to you.

Look at this simple example template:

<cftry>
    Retrieve user information
    <cfcatch> Handle exception </cfcatch>
</cftry>
<cftry>
    Retrieve information about the product’s state
    <cfcatch> Handle exception </cfcatch>
</cftry>
<cftry>
    Send an e-mail with error and additional
     retrieved information
    <cfcatch> Handle exception </cfcatch>
</cftry>

If retrieving user information fails, we will still be sent an e-mail containing the error information and information about the product’s state. This is exactly the isolation we are going for. Subcomponents within in the try-catch blocks can be wrapped in their own try-catch blocks as well, creating an even more granular and fault-tolerant template.

When errors with your Site-wide Error Hander are caught, be sure you attempt to report information on them instead of letting them get swallowed up silently. In the above example, the information about the error encountered while retrieving user information should be included in the final e-mail sent. This feedback is critical in debugging and improving the Site-wide Error Handler—you will be glad you have it. However, be sure to wrap this reporting in its own try-catch in case there is an error with reporting the error. The overall theme is: the more paranoid the better!

By treating your Site-wide Error Handler as a series of granular tasks contained within try-catch blocks, the effect of failures will be limited while the maximum amount of error information safely makes its way back to you. This will be speed up the time it takes to identify, diagnose, and fix bugs and thus improve the quality of your software. <cftry> and <cfcatch> tags are essentially free, but engineering time is not.