Selenium onLoad() handling

At APT, we’ve recently completed a transition to using Selenium’s Java RC¹ for our regression testing framework. One of the excellent things about Selenium RC is the ability to add functionality with relative ease.

In this entry, I want to briefly cover how we get around the onLoad() event caveat of Selenium. Since Selenium works by Javascript injection, it is unable to catch onLoad() events, which can prove to be troublesome when dialog boxes are hit onLoad(). Our code base is relatively large for such a small company and in testing we are really pushing the limits of Selenium. Our goal is to have all of our front-end testing done through Selenium, which means that it needs to elegantly handle Javascript errors and onLoad confirmations/alerts.

When executing an action such as click(), Selenium can hang if a Javascript error is hit. In order to handle this, we now spawn a separate thread to handle any action that might alter the state of the page. The main thread is then able to poll and check if the action thread has hung and if it has we call an AutoIt script² which closes the dialog box and logs an error to our database. This is especially powerful because it lets our tests continue to run with virtually no human interaction.

By leveraging anonymous classes in Java, any Selenium function can quickly be overloaded and run in a custom multithreaded execution thread, as well as make any encountered errors available for the main thread to handle.

This is just an example of how we are automating Selenium, which is crucial in order for it to work with our distributed framework. In my next blog entry I would like to cover how we are accomplishing quick distributed compilation.

- Dan Hackner

1 – http://seleniumhq.org/projects/remote-control/

2 – a Window’s GUI level scripting language. http://www.autoitscript.com/autoit3/

Why am I getting a SyntaxError when trying to start Apache after adding the PHP module?

This one was a killer to figure out and I can't believe I couldn't find the answer with Google. I just installed Apache 2.11 and PHP 5.2.8. Both worked independent of each other, but when I tried to add the PHP module to Apache, it wouldn't start. Specifically, I added:


LoadModule php5_module "c:/path/to/php/php5apache2.dll"
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps


to the httpd.conf file. This was what every set of setup instructions said to do. When I originally tried to fire up Apache as service, it couldn't start. To get some information about the error I tried to start it from the command line. This is probably obvious to most, but you can of course run the web server from the command line. Something like:


>>> c:\Program Files\Apache\bin\httpd.exe


ought to do the trick. Obviously, if you install it elsewhere, replace the relevant parts of the path.

Anyway, I ran it from the command line and got the terrific error message:


Syntax error in line 128 of C:/Program Files/Apache/conf/httpd.conf: Cannot load c:/path/to/php/php5apache2.dll in to server: The specified module could not be found.


It's a little misleading that it says "SyntaxError" because I then spent about 15 minutes trying to figure out if I had a typo somewhere. When I finally started Googling around, I quickly realized the important part of the error message was the last sentence.

To make a long story short, I finally was tipped off by the fact someone mentioned using Apache 2.0.x instead of Apache 2.2 to solve the problem. Ah ha! If you look in the PHP installation directory, there is a suspicious DLL named php5apache2_2.dll. I switched my httpd.conf to use that one instead, and bingo. We're in business.

Like my last post, hopefully this helps someone else stuck in this situation. I had a feeling there might be something about different versions of Apache when I was downloading it and my options were 2.11 and 2.0.63. I didn't notice any discussion of what the big differences were on the Apache site, but apparently there are some. Feel free to leave a comment about the main differences between the versions if you know.

After I install Apache, why am I forbidden from hitting any page in my webroot?

Now there are a lot of possible answers to this question, but I searched Google for quite some time looking for answers. Everyone mentioned checking the permissions on the folder, which is good advice, you should start there. But I don't know much about permission and everything seemed to look okay and still nothing. Since I was migrating from IIS to Apache, I already had my webroot setup to work with IIS and that worked fine, so I was a bit at a loss for what was wrong. Well, the problem I had at least ended up being in the httpd.conf file. Of course I had changed my DocumentRoot to:

DocumentRoot "C:/www"

but it turns out there is another place in the httpd.conf where you need to update the webroot about 30 lines down. There was a nice comment above the code that said it needed to be changed, so this is yet another reason to at least skim through the configuration file. Anyway, I edited that line and voila, everything works. Here in the line I edited along with the comment above it:


#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#

#
# This should be changed to whatever you set DocumentRoot to.
#
<Directory "C:/www">


Here's to hoping this saves someone some time and also a reminder to read through the configuration or ini files once. Yes, it's tedious and often boring, but you always learn something and it can keep you from wasting an hour like I did.

Also, if anyone is wondering, I'm running Apache 2.11. It's probably a good idea to search through the httpd.conf file for references to "htdocs" since that's where the default webroot is located in case future versions include references to your document root elsewhere.

War rooms!

APT wants their new recruits to pick up things fast. Very fast. A small company needs to keep progressing and there isn't too much time to get everyone up to speed.

So how does someone who has never even typed any html or “public static void main” cope with this situation when they join?

APT's solution = the war room. The “war room” is what we’ve termed a room filled with about 5 engineers at close quarters. While this may sound unappealing at first, it is actually awesome to have your team mates right by you. Nothing beats learning by example, and even learning by osmosis by overhearing conversations among other engineers.

Having ready and willing help literally right by your side rapidly accelerates the learning of new engineers as they never have to spin their wheels waiting for assistance when they get stuck. You can get someone to look at your code with a yell, a nudge or the highly effective throw of a stuffed gorilla (because most people listen to music at work and might not hear you yell).

It is a fun and exciting environment to learn in and if you like cooperation and team work, you'd much rather have this that a lot of space and your own private office. In fact, our Senior VP of Engineering leaves his own office empty so that he can join the rest of his team members in one of our war rooms.

Multi-part emails on the iPhone and Thunderbird

At APT, most of us read our email in Outlook. But many of us also have iPhones, and a few of the more adventurous folks (myself included) use Thunderbird out of the office. For the longest time, our software's emails would look just fine in Outlook but would be completely blank in other mail clients (notably iPhone and Thunderbird).

So it turns out that when you send a multi-part email, and one part is plain text and the other part is HTML, most clients will display whatever part occurs LAST.

When you use the <cfmail> tag with <cfmailpart>, and the only part you declare is HTML, ColdFusion "helpfully" adds another plain text mailpart for you. At the end. With no content. Thanks a lot, ColdFusion. Very helpful.

So, the "solution" is to explicitly declare an empty <cfmailpart type="text"> tag BEFORE the <cfmailpart type="html"> tag.

Automatic Bug Filing

I like to automate things. This is welcome trait at APT, as rapidly developing software with an engineering team about 20 in size does not leave much time for manual testing. Out of necessity, we have built a fairly sophisticated automating testing framework, which has been critical in monitoring the integrity of our code. We have software that interacts with our product as if somebody was controlling it themselves. Along the way it checks for errors, or even worse, changed output numbers. The testing code that tells the software what to do is dynamically generated from an object-oriented state-based model abstracted in a database. This allows us to quickly create thousands of test cases that interact with our product in a variety of different ways. These test cases are prioritized and assigned to one of about a dozen automated testing machines which constantly execute them every minute of every day of every week and report on the results.

So there we have it: a distributed prioritized automated testing framework. What more could we want? Well, I found myself spending a lot of time examining the failed tests. If I determined that the problem encountered was not a known issue, I would file a bug a report with the relevant information. Otherwise, I would have to take note that we know about this issue and ignore that test until it is fixed. My coworkers were experiencing the same thing. As we scaled our framework to run more and more tests, we had no analogous expansion of our abilities to monitor and react to the result of these tests. This is where our affinity for automating things comes in. Why not automate responses to our automated tests? And this is exactly what we did.

Now when one of our automated tests hit an error, a check is done to see if it is a new error or not. If it is not a new error, we associate the test with it. This association helps us avoid wasting any more time on subsequent failures as well as logging which tests to use to determine if the problem has been fixed. If it is a new error, we automatically file a detailed bug report with an appropriate priority determined by characteristics of the test case and the error hit.

This automation was not without complexities; in fact we are still working out some kinks. First of all, it hinges on the ability to accurately determine if errors are new or not. Once that is done, you want to be able to filter out errors that are not relevant. Automatic bug filing is a fine line. File too few bugs and you still must spend time going over reports checking for things that may have been missed. File too many bugs and you have to go through them all and weed out the legitimate ones. However once the logic is tweaked the previously manual task of responding to the results of automated tests is now automated itself. The benefits include less time spent looking over reports, as well as zero lag time between the time a problem occurs and the time a bug report is filed, quickly bringing the issue to the attention of product engineers. And of course there is the good feeling you get when you’ve automated something that used to be done manually!

Starting Selenium Server in Java

For some of our automated tests we are switching to use the open-source project Selenium-RC. You can read more about it at its web site: http://selenium-rc.openqa.org/, but essentially it runs a java server which can control an internet browser, and then your testing code sends commands to this server. One key part of this setup is that you need the server running while your testing code is executing. For automated testing machines it would be no big deal to make the Selenium server a service; however developers probably don’t want it running all the time—in fact they do not want to think about it!

Thus our solution was to have our testing code launch the server. I’ve seen a number of posts on various forums asking how to start the selenium server form Java, but none of them had concrete answers. Thus I will reproduce our implementation for you to use and modify as you please:

Process p = null;

try {

    String[] cmd = {"java","-jar","C:\\<path to selenium>\\server\\selenium-server.jar" };

    p = Runtime.getRuntime().exec(cmd);

} catch (IOException e) {

    System.out.println("IOException caught: "+e.getMessage());

    e.printStackTrace();

}


System.out.println("Waiting for server...");

int sec = 0;

int timeout = 20;

boolean serverReady = false;

try {

    BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));

    while (sec < timeout && !serverReady) {

        while (input.ready()) {

            String line = input.readLine();

            System.out.println("From selenium: "+line);

            if (line.contains("Started HttpContext[/,/]")) {

                serverReady = true;

            }

    }

        Thread.sleep(1000);

         ++sec;

    }

    input.close();

} catch (Exception e) {

    System.out.println("Exception caught: "+e.getMessage());

}



if (!serverReady) {

    throw new RuntimeException("Selenium server not ready");

}



System.out.println("Done waiting");

Some notes on the above code:

I left in some handy print statements; however these are of course completely optional.

For non-automated testing machines, be sure to have your outer most try-catch block of your testing code kill the server or it may be left running even when the testing code finishes.

For code on automated testing machines, you may want to check to see if the server is running and start it only if it is not. This way you don’t waste time waiting for the server to be ready if another test already brought it up.