I like to automate things. This is welcome trait at APT, as rapidly developing software with an engineering team about 20 in size does not leave much time for manual testing. Out of necessity, we have built a fairly sophisticated automating testing framework, which has been critical in monitoring the integrity of our code. We have software that interacts with our product as if somebody was controlling it themselves. Along the way it checks for errors, or even worse, changed output numbers. The testing code that tells the software what to do is dynamically generated from an object-oriented state-based model abstracted in a database. This allows us to quickly create thousands of test cases that interact with our product in a variety of different ways. These test cases are prioritized and assigned to one of about a dozen automated testing machines which constantly execute them every minute of every day of every week and report on the results.
So there we have it: a distributed prioritized automated testing framework. What more could we want? Well, I found myself spending a lot of time examining the failed tests. If I determined that the problem encountered was not a known issue, I would file a bug a report with the relevant information. Otherwise, I would have to take note that we know about this issue and ignore that test until it is fixed. My coworkers were experiencing the same thing. As we scaled our framework to run more and more tests, we had no analogous expansion of our abilities to monitor and react to the result of these tests. This is where our affinity for automating things comes in. Why not automate responses to our automated tests? And this is exactly what we did.
Now when one of our automated tests hit an error, a check is done to see if it is a new error or not. If it is not a new error, we associate the test with it. This association helps us avoid wasting any more time on subsequent failures as well as logging which tests to use to determine if the problem has been fixed. If it is a new error, we automatically file a detailed bug report with an appropriate priority determined by characteristics of the test case and the error hit.
This automation was not without complexities; in fact we are still working out some kinks. First of all, it hinges on the ability to accurately determine if errors are new or not. Once that is done, you want to be able to filter out errors that are not relevant. Automatic bug filing is a fine line. File too few bugs and you still must spend time going over reports checking for things that may have been missed. File too many bugs and you have to go through them all and weed out the legitimate ones. However once the logic is tweaked the previously manual task of responding to the results of automated tests is now automated itself. The benefits include less time spent looking over reports, as well as zero lag time between the time a problem occurs and the time a bug report is filed, quickly bringing the issue to the attention of product engineers. And of course there is the good feeling you get when you’ve automated something that used to be done manually!
Thursday, October 23, 2008
Monday, October 6, 2008
Starting Selenium Server in Java
For some of our automated tests we are switching to use the open-source project Selenium-RC. You can read more about it at its web site: http://selenium-rc.openqa.org/, but essentially it runs a java server which can control an internet browser, and then your testing code sends commands to this server. One key part of this setup is that you need the server running while your testing code is executing. For automated testing machines it would be no big deal to make the Selenium server a service; however developers probably don’t want it running all the time—in fact they do not want to think about it!
Thus our solution was to have our testing code launch the server. I’ve seen a number of posts on various forums asking how to start the selenium server form Java, but none of them had concrete answers. Thus I will reproduce our implementation for you to use and modify as you please:
Process p = null;
try {
String[] cmd = {"java","-jar","C:\\<path to selenium>\\server\\selenium-server.jar" };
p = Runtime.getRuntime().exec(cmd);
} catch (IOException e) {
System.out.println("IOException caught: "+e.getMessage());
e.printStackTrace();
}
System.out.println("Waiting for server...");
int sec = 0;
int timeout = 20;
boolean serverReady = false;
try {
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
while (sec < timeout && !serverReady) {
while (input.ready()) {
String line = input.readLine();
System.out.println("From selenium: "+line);
if (line.contains("Started HttpContext[/,/]")) {
serverReady = true;
}
}
Thread.sleep(1000);
++sec;
}
input.close();
} catch (Exception e) {
System.out.println("Exception caught: "+e.getMessage());
}
if (!serverReady) {
throw new RuntimeException("Selenium server not ready");
}
System.out.println("Done waiting");
Some notes on the above code:
Friday, August 29, 2008
SQL INSERTs
The other day, we were confronted with an interesting SQL problem. A user wanted to be able to paste in a set of record IDs to be saved and made available for use throughout the rest of the product. For large sets of data (in the millions of rows), it makes sense for users to upload these files to us via FTP or even through the web server, then have the files loaded into the database. In these cases, users most likely already have the data readily available in some file format, so it’s easy for them to just upload it to us. But in the small to medium size cases (anywhere from hundreds to tens of thousands of records), the user could be pulling the data from an Excel spreadsheet or similar document so that it’s easier for him or her to simply paste the data into a text field of a form. For simplicity’s sake, we can assume that the data consists of a set of ID numbers that are carriage return delimited.
So then, how do we persist these IDs to the database? Essentially we are given a ginormous string of delimited IDs that we want to dump into a table with a single column that is ID number, like so:
In tackling this problem, we considered four different approaches. Our goal was to find the fastest approach (in terms of user wait time), since in many cases the user could be forced to wait several minutes for the upload to complete. As with many enterprise web applications, our DB resides on a different machine than our web server. So each approach’s performance is really driven by two factors:
Query Time – the total amount of time the DB spends processing the queries that insert IDs into the table we designate
NetworkTransfer Time – the total amount of time spent transferring requests from our web server to the DB
The first approach we tried, which we’ll refer to as the Naïve Insert Loop, was to loop through the delimited IDs, inserting a single row into the database for each ID. Each query to insert a row was fired off as a separate DB request:
This approach is problematic for several reasons, all stemming from the fact that it creates an individual query per record ID and fires it off from the web app to the DB one at a time. Since each query and request has a certain overhead to it, this solution pays huge penalties for the large numbers of queries and requests used. We used this naïve approach as a baseline for which to improve upon.
Recognizing that we needed to reduce the number of queries and requests fired, we then considered Improved Insert Loop, which was very similar to the Insert Loop except that we combined the INSERTs together via UNION ALLs before firing them off to the DB:
We can combine these INSERTs together into batches of a thousand* SELECT statements UNION ALLed together, so we essentially have reduced the number of queries and network requests by a factor of thousand. The queries and requests are themselves approximately a thousand times larger than before. But we have improved net performance because by combining a thousand queries together, we don’t have to pay the overhead attached to all of the individual thousand queries and requests we would have run otherwise. For instance, by reducing the number of queries, we are reducing the number of DB transactions, and therefore we reduce the number of disk writes that happen on the DB since we are reducing the number of transaction log flushes.
In both approaches above, we are forced to wrap the IDs in queries that insert them into the table on the DB. If we could somehow transmit the raw IDs to the DB and have them parsed and inserted completely on the DB side, we could greatly reduce the size of the data sent over the network and thus greatly reduce the Network Transfer Time.
With that in mind, we came up with the Stored Procedure Loop approach. Essentially we would pass the entire string of delimited IDs as a TEXT field to a stored procedure, which would do the work of parsing the field and INSERTing the individual records into a target table. Below is the stored procedure definition. It starts by logging the entirety of the paste request into a dataStagingTable and parses the data logged in the table.
This approach yielded a huge performance improvement, as we had essentially minimized the Network Transfer Time by minimizing the amount of data being transmitted. We could have further improved the Query Time by reducing the number of transactions via either explicit transaction blocks or by combining INSERTs into batches as we did with the Improved Loop Insert. But even by making these improvements, we still would have had approximately the same number of queries being run as we did for the Improved Loop Insert (although now they would be run within the stored procedure on the DB side).
In order to further improve the Query Time, we finally arrived at a fourth approach, the BCP Insert. BCP is a utility included with SQL Server that loads data from a file into a DB table. In this case, we dump the delimited IDs into a text file, then invoke BCP on the new text file. To run the BCP utility, we had to make sure there was a way for the web server to transmit these files to the DB machine. After that, we could run the BCP utility as a console command:
Creating the format file for configuring BCP to parse the data correctly was also straightforward:
Like with the Stored Procedure Loop, the data sent is essentially just the raw data, but in this case, SQL loads all the data without having to run a whole slew of queries. Furthermore, SQL’s loading of this data is not logged, which results in much less overhead than the previous approaches. This final approach does require additional time to move data to and from the file system, but in sum it was still faster than the other approaches. It is important to note, however, that as the number of records being pasted decreases, the difference in performance between these approaches also decreases. In fact, if we are inserting fewer than a thousand records, it is actually faster to revert from BCP Insert to one of the more naïve solutions, since the overhead of the file dump begins to dominate the actual Query and Network Transfer Time.
In conclusion, the BCP Insert solution was the fastest of the four. This is not unexpected, since these are the types of operations that BCP was designed to perform quickly. Another interesting build on these solutions would have been to incorporate a means of compression of the raw data before transmitting from web server to DB server. That said, we found this to be an interesting chance to experiment with a few different creative solutions.
* The thousand here is selected for simplicity; we can continue to increase this number to maximize the benefit of these batch combinations. The cap on the batch size is dependent on the RDBMS you are using.
So then, how do we persist these IDs to the database? Essentially we are given a ginormous string of delimited IDs that we want to dump into a table with a single column that is ID number, like so:
CREATE TABLE records
(
recordID INT NOT NULL,
CONSTRAINT PK_records PRIMARY KEY CLUSTERED (recordID)
)
In tackling this problem, we considered four different approaches. Our goal was to find the fastest approach (in terms of user wait time), since in many cases the user could be forced to wait several minutes for the upload to complete. As with many enterprise web applications, our DB resides on a different machine than our web server. So each approach’s performance is really driven by two factors:
The first approach we tried, which we’ll refer to as the Naïve Insert Loop, was to loop through the delimited IDs, inserting a single row into the database for each ID. Each query to insert a row was fired off as a separate DB request:
INSERT INTO records (recordID)
SELECT 12
INSERT INTO records (recordID)
SELECT 15
INSERT INTO records (recordID)
SELECT 17
...
This approach is problematic for several reasons, all stemming from the fact that it creates an individual query per record ID and fires it off from the web app to the DB one at a time. Since each query and request has a certain overhead to it, this solution pays huge penalties for the large numbers of queries and requests used. We used this naïve approach as a baseline for which to improve upon.
Recognizing that we needed to reduce the number of queries and requests fired, we then considered Improved Insert Loop, which was very similar to the Insert Loop except that we combined the INSERTs together via UNION ALLs before firing them off to the DB:
INSERT INTO records (recordID)
SELECT 12
UNION ALL
SELECT 15
UNION ALL
SELECT 17
...
INSERT INTO records (recordID)
SELECT 27
UNION ALL
SELECT 28
UNION ALL
SELECT 54
...
We can combine these INSERTs together into batches of a thousand* SELECT statements UNION ALLed together, so we essentially have reduced the number of queries and network requests by a factor of thousand. The queries and requests are themselves approximately a thousand times larger than before. But we have improved net performance because by combining a thousand queries together, we don’t have to pay the overhead attached to all of the individual thousand queries and requests we would have run otherwise. For instance, by reducing the number of queries, we are reducing the number of DB transactions, and therefore we reduce the number of disk writes that happen on the DB since we are reducing the number of transaction log flushes.
In both approaches above, we are forced to wrap the IDs in queries that insert them into the table on the DB. If we could somehow transmit the raw IDs to the DB and have them parsed and inserted completely on the DB side, we could greatly reduce the size of the data sent over the network and thus greatly reduce the Network Transfer Time.
With that in mind, we came up with the Stored Procedure Loop approach. Essentially we would pass the entire string of delimited IDs as a TEXT field to a stored procedure, which would do the work of parsing the field and INSERTing the individual records into a target table. Below is the stored procedure definition. It starts by logging the entirety of the paste request into a dataStagingTable and parses the data logged in the table.
CREATE TABLE dataStagingTable (
logID int not null identity(1,1),
data Text,
pastingTime datetime
)
exec processPastedData 'wilfred', '2
3
4
5', '
',','
select * from wilfred
CREATE PROCEDURE dbo.processPastedData
@targetTable VARCHAR(32),
@data TEXT,
@rowDelimiter CHAR,
@colDelimiter CHAR
AS
BEGIN
DECLARE @logID INT
DECLARE @dlen BIGINT
DECLARE @offset INT
DECLARE @linePtr INT
DECLARE @buf varchar(4000)
DECLARE @cols varchar(255)
INSERT INTO dataStagingTable (data, pastingTime)
SELECT @data, getDate()
SELECT @logID = @@Identity
SELECT @offset = 1
SELECT @dlen = datalength(data)
FROM dataStagingTable
WHERE logID = @logID
SELECT @cols = 'recordID'
SET NOCOUNT ON
WHILE (@offset > 0)
BEGIN
SELECT @linePtr = CHARINDEX(@rowDelimiter, SUBSTRING(data, @offset, 4000))
FROM dataStagingTable
WHERE logID = @logID
if (@linePtr > 0)
SELECT @buf = REPLACE (SUBSTRING(data, @offset , @linePtr-1), @colDelimiter, ''',''')
FROM dataStagingTable
WHERE logID = @logID
else
SELECT @buf = REPLACE (SUBSTRING(data, @offset , 8000), @colDelimiter, ''',''')
FROM dataStagingTable
WHERE logID = @logID
SELECT @buf = REPLACE (@buf, char(13), '')
EXEC ('INSERT INTO ' + @targetTable + ' (' + @cols + ') SELECT ' + @buf)
SET @offset = @offset + @linePtr
if (@linePtr = 0)
BREAK
END
END
This approach yielded a huge performance improvement, as we had essentially minimized the Network Transfer Time by minimizing the amount of data being transmitted. We could have further improved the Query Time by reducing the number of transactions via either explicit transaction blocks or by combining INSERTs into batches as we did with the Improved Loop Insert. But even by making these improvements, we still would have had approximately the same number of queries being run as we did for the Improved Loop Insert (although now they would be run within the stored procedure on the DB side).
In order to further improve the Query Time, we finally arrived at a fourth approach, the BCP Insert. BCP is a utility included with SQL Server that loads data from a file into a DB table. In this case, we dump the delimited IDs into a text file, then invoke BCP on the new text file. To run the BCP utility, we had to make sure there was a way for the web server to transmit these files to the DB machine. After that, we could run the BCP utility as a console command:
bcp ClientDB.dbo.records in dataDump.txt -S DBMachine -U userID -P password -f formatFile.fmt
------------
ClientDB.dbowner.records = [DATABASE].[SCHEMA].[TABLE TO UPLOAD DATA TO]
dataDump.txt = data file that contains pasted data
-S DBMachine = name of server to connect to
-U userID = tells bcp to use a specific userID to log into DB machine
-P password = tells bcp the password to use with userID to log into DB machine
-f formatFile.fmt = format file that tells bcp how to parse the data file and how to insert into records table
Creating the format file for configuring BCP to parse the data correctly was also straightforward:
8.0
1
1 SQLCHAR 0 50 "\r\n" 1 recordID SQL_Latin1_General_CP1_CI_AS
------------
8.0 = SQL Version
1 = number of columns
Third row going left to right:
1 = File field order
SQLCHAR = Host file data type
0 = Prefix length
50 = Host file data length
"\r\n" = line terminator
1 = Server column order
recordID = name of column we are uploading to
SQL_Latin1_General_CP1_CI_AS = column collation type
Like with the Stored Procedure Loop, the data sent is essentially just the raw data, but in this case, SQL loads all the data without having to run a whole slew of queries. Furthermore, SQL’s loading of this data is not logged, which results in much less overhead than the previous approaches. This final approach does require additional time to move data to and from the file system, but in sum it was still faster than the other approaches. It is important to note, however, that as the number of records being pasted decreases, the difference in performance between these approaches also decreases. In fact, if we are inserting fewer than a thousand records, it is actually faster to revert from BCP Insert to one of the more naïve solutions, since the overhead of the file dump begins to dominate the actual Query and Network Transfer Time.
In conclusion, the BCP Insert solution was the fastest of the four. This is not unexpected, since these are the types of operations that BCP was designed to perform quickly. Another interesting build on these solutions would have been to incorporate a means of compression of the raw data before transmitting from web server to DB server. That said, we found this to be an interesting chance to experiment with a few different creative solutions.
* The thousand here is selected for simplicity; we can continue to increase this number to maximize the benefit of these batch combinations. The cap on the batch size is dependent on the RDBMS you are using.
Thursday, August 14, 2008
Debugging at APT - Part 3: Fiddler
Fiddler has been around for a while, but I was only introduced to it a few months ago. Earlier this year, we were able to budget some time toward improving our software's performance, rather than continuing to add new features as quickly as we had been. Fiddler was especially useful for telling us just how much work our web servers (and our clients' browsers) were doing, and where some of our bottlenecks and improvement opportunities were.
In my first post I mentioned that we have some pretty complex pages throughout our software. Before optimizing them, we were making hundreds of HTTP requests per page (including the now-infamous 85 iframes), and we were applying the same super-strict "no cache" rules to our static files that we applied to our dynamically-generated content. As a result, even our simplest pages required well over 100K of HTTP traffic, and some of the most complex pages were closer to 2MB. Fiddler's biggest strength for us was seeing how each of the various caching directives you can send in HTTP headers affects the browser and network traffic.
Coincidentally, right after I learned about Fiddler, I was asked to help troubleshoot a performance concern raised by one of our newer clients. They were sporadically and unpredictably experiencing "This page cannot be displayed (cannot find server or DNS error)" messages. It was inconsistent but seemed to happen primarily during the client's core business hours, and none of our other clients were experiencing comparable symptoms, so we figured it could be a network problem on their end. A combination of Fiddler (on one client user's computer) and Wireshark (on our web server) showed us a pretty clear pattern of dropped and resubmitted requests at the HTTP level. At the same time, our own internal logging showed an extraordinarily high variance in round-trip performance.
It turned out that the client's internal firewall was sporadically dropping some outgoing requests and incoming responses during times of peak traffic on their general-purpose network. The solution was to establish a VPN tunnel, using a separate infrastructure that they already had in place for their vendors and business partners. In hindsight, we might have reached the same conclusion even more quickly if we had run Fiddler on both sides and compared the traces.
In my first post I mentioned that we have some pretty complex pages throughout our software. Before optimizing them, we were making hundreds of HTTP requests per page (including the now-infamous 85 iframes), and we were applying the same super-strict "no cache" rules to our static files that we applied to our dynamically-generated content. As a result, even our simplest pages required well over 100K of HTTP traffic, and some of the most complex pages were closer to 2MB. Fiddler's biggest strength for us was seeing how each of the various caching directives you can send in HTTP headers affects the browser and network traffic.
Coincidentally, right after I learned about Fiddler, I was asked to help troubleshoot a performance concern raised by one of our newer clients. They were sporadically and unpredictably experiencing "This page cannot be displayed (cannot find server or DNS error)" messages. It was inconsistent but seemed to happen primarily during the client's core business hours, and none of our other clients were experiencing comparable symptoms, so we figured it could be a network problem on their end. A combination of Fiddler (on one client user's computer) and Wireshark (on our web server) showed us a pretty clear pattern of dropped and resubmitted requests at the HTTP level. At the same time, our own internal logging showed an extraordinarily high variance in round-trip performance.
It turned out that the client's internal firewall was sporadically dropping some outgoing requests and incoming responses during times of peak traffic on their general-purpose network. The solution was to establish a VPN tunnel, using a separate infrastructure that they already had in place for their vendors and business partners. In hindsight, we might have reached the same conclusion even more quickly if we had run Fiddler on both sides and compared the traces.
Thursday, July 31, 2008
Debugging at APT - Part 2: IE Developer Toolbar
As I said last time, we don't officially support Firefox yet because our clients use it so rarely. So we were relieved when Microsoft finally started to catch up by releasing the IE Developer Toolbar. We're often frustrated by an inexplicable error (either an actual JS error, or just ugly/broken HTML/CSS) that vexes us in IE but works perfectly fine in Firefox. For the JavaScript errors, we've had the Visual Studio Script Debugger available to us for a while (once we jumped through the necessary hoops to enable it), but sometimes the problem is that some of our JS code just isn't being loaded or executed properly. (Never mind that Script Debugger often shows JS exceptions in the middle of completely unrelated HTML markup.) Since we get some of our JS from asynchronous AJAX calls, it's something that View Source wouldn't be able to tell us. Once again, "select element by click" (equivalent to "Inspect" in Firebug) has been a great friend.
I also like the syntax-highlighted View Source, although I usually end up using something like Notepad++ to view source from a browser. But even better, Developer Toolbar's View Source includes the list of CSS declarations that apply to the portion of the page I've selected.
As you can see above and below, I have selected one element buried pretty deep within one of our pages, and I can easily use "Element Source with Style" to see not only its entire innerHTML but also all the CSS declarations, where they came from, and all parent tags' IDs and styles.
Next time I'll talk about Fiddler, also owned by Microsoft, which helps us debug "over the wire."
I also like the syntax-highlighted View Source, although I usually end up using something like Notepad++ to view source from a browser. But even better, Developer Toolbar's View Source includes the list of CSS declarations that apply to the portion of the page I've selected.
As you can see above and below, I have selected one element buried pretty deep within one of our pages, and I can easily use "Element Source with Style" to see not only its entire innerHTML but also all the CSS declarations, where they came from, and all parent tags' IDs and styles.
Next time I'll talk about Fiddler, also owned by Microsoft, which helps us debug "over the wire."
Friday, July 25, 2008
Javascript Basics: The second and third functions you should write and use all the time
After an embarrassingly long delay, I've decided to put two functions I use all the time in this post to get us somewhat back on track. I also plan on including some other posts about our experimentation with the Yahoo! User Interface (YUI) library and the jQuery library I mentioned last time.
But anyways, back to this post's topic, javascript functions you should write for yourself and be using all the time. This weeks functions are actually pretty simple, but I've talked to and helped a lot of people who weren't using them or had written them so that they're not quite as generic as they should be.
One of the main uses of javascript is to interact with the document object model (DOM) created by the browser. At the most basic level, the ability to take static HTML and manipulate not only the properties of those HTML elements, but also remove those elements as well as create new elements really gives javascript a tremendous amount of power.
Quick digression: Of course, this isn't the only thing javascript does for us. Javascript also plays a key role in asynchronous javascript and XML (AJAX), which has revolutionized the way a lot of users interact with web pages. Some would probably argue that AJAX is even more important than the ability to create dynamic HTML (DHTML). I think they're both critical features, and which you think is more important probably will depend a lot on the web site or application you're developing.
Regardless of whether manipulating the DOM is the most important feature of javascript, everyone can admit it's extremely powerful. One of the most frequent manipulations that developers use javascript for is showing and hiding HTML elements. On pretty much every web page there is some extra or more detailed information that you might not want to display to the user immediately, but would like to reveal when they click a link or check a checkbox. Showing the user only the information they need to know at any particular time can greatly increase the usability of a site.
Here is a simple function that will automatically display an element based on its ID:
Notice that this function builds off of the $() function from the first post that will get an HTML element based on its ID. It gets a reference to that element, accesses the style attribute of that element, and finally sets the display property of the style attribute to the empty string. At this point you might have a couple questions worth addressing.
First, it's worth pointing out that the style attribute of an element is not a simple value as it is for an element's ID or class. In this case, the style attribute is an object because there are a lot of styles that might apply to a particular element such as the font size, positioning on the page, or background color. All of these style properties could be stored as one big string, but it's pretty easy to see how that would be a nightmare for developers (editing an element's "font-size" wouldn't be very easy).
The other two questions worth answering are why did we choose to change the display property and why did we set it to the empty string? These are both good questions. There are two style properties that control whether an item can be seen or not, display and visibility. The visibility property can have a value of "hidden". It might seem like this would do what we want, but what the visibility property actually does is changes whether an element is visible without removing the space it takes up from the page. This means if a I have a paragraph element whose visibility I set to hidden, I won't be able to see that paragraph element, but I will see a big white space where that element should be. This is rarely what you want, but sometimes it is. Usually what you want is for that element to take up space on the page when it is visible or displayed, but be removed from the flow or layout of the page when it is not being displayed. This is exactly what the display property does. When the display property for an element is set to "none", then that element is completely removed from the flow of the page so there is no unnecessary white space.
So the only question remaining is why did we set the display property to the empty string in order to get our element to appear. If anything, an empty string might indicate that something shouldn't be displayed. In this case, as is the case with a lot of the style properties, it is set to the default value for that property. It just so happens that the default value for the display property is whatever display value for that element will display it properly. This means for block elements like <div> or <p> elements, the display value will be set to "block". For inline elements such as <strong> or <span> elements, it will be "inline". Certain elements, such as elements, even have different display types in different browsers. In Internet Explorer, a <tr> is displayed properly with the value of "block", but needs to be set to "table-row" in Firefox. This is where setting the display value to the empty string and allowing the browser to then apply the default really shines.
So at this point it should be pretty clear why this function was written this way (but if it isn't, let me know in the comments). It's nice to have all of these somewhat complex choices encapsulated in a single easy to understand function that anyone can use, not to mention there's a lot less typing to do.
So what is the other function for this post?
I don't think this one needs much explanation, so here it is:
I don't think there's anything I need to say about this function. The only slightly interesting thing to note here is that, unlike displaying elements, if you don't want display elements, setting their display value to "none" always does the trick.
Cool, so that's it for this post. Add these functions to wherever you put the $() function -- hopefully an external javascript file that you include on pages you need javascript on. Then use them in your javascript and see how much shorter and easier to understand your javascript becomes!
But anyways, back to this post's topic, javascript functions you should write for yourself and be using all the time. This weeks functions are actually pretty simple, but I've talked to and helped a lot of people who weren't using them or had written them so that they're not quite as generic as they should be.
2. show()
One of the main uses of javascript is to interact with the document object model (DOM) created by the browser. At the most basic level, the ability to take static HTML and manipulate not only the properties of those HTML elements, but also remove those elements as well as create new elements really gives javascript a tremendous amount of power.
Quick digression: Of course, this isn't the only thing javascript does for us. Javascript also plays a key role in asynchronous javascript and XML (AJAX), which has revolutionized the way a lot of users interact with web pages. Some would probably argue that AJAX is even more important than the ability to create dynamic HTML (DHTML). I think they're both critical features, and which you think is more important probably will depend a lot on the web site or application you're developing.
Regardless of whether manipulating the DOM is the most important feature of javascript, everyone can admit it's extremely powerful. One of the most frequent manipulations that developers use javascript for is showing and hiding HTML elements. On pretty much every web page there is some extra or more detailed information that you might not want to display to the user immediately, but would like to reveal when they click a link or check a checkbox. Showing the user only the information they need to know at any particular time can greatly increase the usability of a site.
Here is a simple function that will automatically display an element based on its ID:
function show(id) {
$(id).style.display = "";
}
Notice that this function builds off of the $() function from the first post that will get an HTML element based on its ID. It gets a reference to that element, accesses the style attribute of that element, and finally sets the display property of the style attribute to the empty string. At this point you might have a couple questions worth addressing.
First, it's worth pointing out that the style attribute of an element is not a simple value as it is for an element's ID or class. In this case, the style attribute is an object because there are a lot of styles that might apply to a particular element such as the font size, positioning on the page, or background color. All of these style properties could be stored as one big string, but it's pretty easy to see how that would be a nightmare for developers (editing an element's "font-size" wouldn't be very easy).
The other two questions worth answering are why did we choose to change the display property and why did we set it to the empty string? These are both good questions. There are two style properties that control whether an item can be seen or not, display and visibility. The visibility property can have a value of "hidden". It might seem like this would do what we want, but what the visibility property actually does is changes whether an element is visible without removing the space it takes up from the page. This means if a I have a paragraph element whose visibility I set to hidden, I won't be able to see that paragraph element, but I will see a big white space where that element should be. This is rarely what you want, but sometimes it is. Usually what you want is for that element to take up space on the page when it is visible or displayed, but be removed from the flow or layout of the page when it is not being displayed. This is exactly what the display property does. When the display property for an element is set to "none", then that element is completely removed from the flow of the page so there is no unnecessary white space.
So the only question remaining is why did we set the display property to the empty string in order to get our element to appear. If anything, an empty string might indicate that something shouldn't be displayed. In this case, as is the case with a lot of the style properties, it is set to the default value for that property. It just so happens that the default value for the display property is whatever display value for that element will display it properly. This means for block elements like <div> or <p> elements, the display value will be set to "block". For inline elements such as <strong> or <span> elements, it will be "inline". Certain elements, such as
So at this point it should be pretty clear why this function was written this way (but if it isn't, let me know in the comments). It's nice to have all of these somewhat complex choices encapsulated in a single easy to understand function that anyone can use, not to mention there's a lot less typing to do.
So what is the other function for this post?
3. hide()
I don't think this one needs much explanation, so here it is:
function hide(id) {
$(id).style.display = "none";
}
I don't think there's anything I need to say about this function. The only slightly interesting thing to note here is that, unlike displaying elements, if you don't want display elements, setting their display value to "none" always does the trick.
Cool, so that's it for this post. Add these functions to wherever you put the $() function -- hopefully an external javascript file that you include on pages you need javascript on. Then use them in your javascript and see how much shorter and easier to understand your javascript becomes!
Monday, July 21, 2008
Debugging at APT - Part 1: Firebug
Like most developers (using any language or platform), folks at APT crave robust and powerful debugging tools. Fortunately, ColdFusion 8 introduced some fantastic new debugging and monitoring tools, on top of the robust logging and request debugging that has been available for a long time. Unfortunately, our architecture seems to prevent us from using most of those tools. (But we'll have plenty of architecture discussions later.) More generally, our system is not "pure" ColdFusion -- we also use Java directly, as well as SQL, JavaScript, CSS, .NET, VBScript, Ant... you name it -- so we often wonder which debugging tools are best for which jobs.
I recently hosted an internal "Wednesday Noon Session" on debugging at APT. (Every three weeks or so, one of our engineers puts together a presentation that focuses on one aspect of development at APT.) I learned a lot by preparing my session, so I'm going to post a series of articles on the tools we use and how we use them. I'll start today with Firebug.
Firebug
For the client side, as well as some server-side debugging, Firebug is one of our best friends. We don’t officially support Firefox for our clients, because their IT departments pretty much all mandate IE6 and prohibit the installation of any other browser. In fact, 95% of our product logins come from IE6, and almost all of the rest are from IE7. Once in a while we even see someone try to use IE6 on Win2K, which typically is missing the past five years worth of bug fixes and security patches. (Yes, even in mid-2008!)
But we’re working toward full Firefox support for a number of reasons, one of which is debugging capability. We have all sorts of complex pages in our software. If you’re not familiar with them, simply finding an element on the page, let alone debugging a script, can be daunting. Thank goodness for the "Inspect" button.
We make heavy use of the Script tab to evaluate JS expressions on-the-fly and step through functions, as well as the Net tab to see what's going on with our AJAX requests (like sorting, searching and pagination within each component). We also used Yahoo!'s YSlow extension to discover that our menus were rendering upwards of 85 iframe tags on each page. (Ouch! Well, it turns out that all of the iframes were there to work around a well-known IE6 bug. But we've since gotten smarter about the "hidden iframe hack" and we've also switched to using YUI menus, which are also smarter about using iframes and about rendering efficiency in general.)
Next time: Firebug's cousin and competitor, the IE Developer Toolbar.
I recently hosted an internal "Wednesday Noon Session" on debugging at APT. (Every three weeks or so, one of our engineers puts together a presentation that focuses on one aspect of development at APT.) I learned a lot by preparing my session, so I'm going to post a series of articles on the tools we use and how we use them. I'll start today with Firebug.
Firebug
For the client side, as well as some server-side debugging, Firebug is one of our best friends. We don’t officially support Firefox for our clients, because their IT departments pretty much all mandate IE6 and prohibit the installation of any other browser. In fact, 95% of our product logins come from IE6, and almost all of the rest are from IE7. Once in a while we even see someone try to use IE6 on Win2K, which typically is missing the past five years worth of bug fixes and security patches. (Yes, even in mid-2008!)
But we’re working toward full Firefox support for a number of reasons, one of which is debugging capability. We have all sorts of complex pages in our software. If you’re not familiar with them, simply finding an element on the page, let alone debugging a script, can be daunting. Thank goodness for the "Inspect" button.
We make heavy use of the Script tab to evaluate JS expressions on-the-fly and step through functions, as well as the Net tab to see what's going on with our AJAX requests (like sorting, searching and pagination within each component). We also used Yahoo!'s YSlow extension to discover that our menus were rendering upwards of 85 iframe tags on each page. (Ouch! Well, it turns out that all of the iframes were there to work around a well-known IE6 bug. But we've since gotten smarter about the "hidden iframe hack" and we've also switched to using YUI menus, which are also smarter about using iframes and about rendering efficiency in general.)
Next time: Firebug's cousin and competitor, the IE Developer Toolbar.
Subscribe to:
Posts (Atom)