At APT, most of us read our email in Outlook. But many of us also have iPhones, and a few of the more adventurous folks (myself included) use Thunderbird out of the office. For the longest time, our software's emails would look just fine in Outlook but would be completely blank in other mail clients (notably iPhone and Thunderbird).
So it turns out that when you send a multi-part email, and one part is plain text and the other part is HTML, most clients will display whatever part occurs LAST.
When you use the <cfmail> tag with <cfmailpart>, and the only part you declare is HTML, ColdFusion "helpfully" adds another plain text mailpart for you. At the end. With no content. Thanks a lot, ColdFusion. Very helpful.
So, the "solution" is to explicitly declare an empty <cfmailpart type="text"> tag BEFORE the <cfmailpart type="html"> tag.
Tuesday, December 16, 2008
Thursday, October 23, 2008
Automatic Bug Filing
I like to automate things. This is welcome trait at APT, as rapidly developing software with an engineering team about 20 in size does not leave much time for manual testing. Out of necessity, we have built a fairly sophisticated automating testing framework, which has been critical in monitoring the integrity of our code. We have software that interacts with our product as if somebody was controlling it themselves. Along the way it checks for errors, or even worse, changed output numbers. The testing code that tells the software what to do is dynamically generated from an object-oriented state-based model abstracted in a database. This allows us to quickly create thousands of test cases that interact with our product in a variety of different ways. These test cases are prioritized and assigned to one of about a dozen automated testing machines which constantly execute them every minute of every day of every week and report on the results.
So there we have it: a distributed prioritized automated testing framework. What more could we want? Well, I found myself spending a lot of time examining the failed tests. If I determined that the problem encountered was not a known issue, I would file a bug a report with the relevant information. Otherwise, I would have to take note that we know about this issue and ignore that test until it is fixed. My coworkers were experiencing the same thing. As we scaled our framework to run more and more tests, we had no analogous expansion of our abilities to monitor and react to the result of these tests. This is where our affinity for automating things comes in. Why not automate responses to our automated tests? And this is exactly what we did.
Now when one of our automated tests hit an error, a check is done to see if it is a new error or not. If it is not a new error, we associate the test with it. This association helps us avoid wasting any more time on subsequent failures as well as logging which tests to use to determine if the problem has been fixed. If it is a new error, we automatically file a detailed bug report with an appropriate priority determined by characteristics of the test case and the error hit.
This automation was not without complexities; in fact we are still working out some kinks. First of all, it hinges on the ability to accurately determine if errors are new or not. Once that is done, you want to be able to filter out errors that are not relevant. Automatic bug filing is a fine line. File too few bugs and you still must spend time going over reports checking for things that may have been missed. File too many bugs and you have to go through them all and weed out the legitimate ones. However once the logic is tweaked the previously manual task of responding to the results of automated tests is now automated itself. The benefits include less time spent looking over reports, as well as zero lag time between the time a problem occurs and the time a bug report is filed, quickly bringing the issue to the attention of product engineers. And of course there is the good feeling you get when you’ve automated something that used to be done manually!
So there we have it: a distributed prioritized automated testing framework. What more could we want? Well, I found myself spending a lot of time examining the failed tests. If I determined that the problem encountered was not a known issue, I would file a bug a report with the relevant information. Otherwise, I would have to take note that we know about this issue and ignore that test until it is fixed. My coworkers were experiencing the same thing. As we scaled our framework to run more and more tests, we had no analogous expansion of our abilities to monitor and react to the result of these tests. This is where our affinity for automating things comes in. Why not automate responses to our automated tests? And this is exactly what we did.
Now when one of our automated tests hit an error, a check is done to see if it is a new error or not. If it is not a new error, we associate the test with it. This association helps us avoid wasting any more time on subsequent failures as well as logging which tests to use to determine if the problem has been fixed. If it is a new error, we automatically file a detailed bug report with an appropriate priority determined by characteristics of the test case and the error hit.
This automation was not without complexities; in fact we are still working out some kinks. First of all, it hinges on the ability to accurately determine if errors are new or not. Once that is done, you want to be able to filter out errors that are not relevant. Automatic bug filing is a fine line. File too few bugs and you still must spend time going over reports checking for things that may have been missed. File too many bugs and you have to go through them all and weed out the legitimate ones. However once the logic is tweaked the previously manual task of responding to the results of automated tests is now automated itself. The benefits include less time spent looking over reports, as well as zero lag time between the time a problem occurs and the time a bug report is filed, quickly bringing the issue to the attention of product engineers. And of course there is the good feeling you get when you’ve automated something that used to be done manually!
Monday, October 6, 2008
Starting Selenium Server in Java
For some of our automated tests we are switching to use the open-source project Selenium-RC. You can read more about it at its web site: http://selenium-rc.openqa.org/, but essentially it runs a java server which can control an internet browser, and then your testing code sends commands to this server. One key part of this setup is that you need the server running while your testing code is executing. For automated testing machines it would be no big deal to make the Selenium server a service; however developers probably don’t want it running all the time—in fact they do not want to think about it!
Thus our solution was to have our testing code launch the server. I’ve seen a number of posts on various forums asking how to start the selenium server form Java, but none of them had concrete answers. Thus I will reproduce our implementation for you to use and modify as you please:
Process p = null;
try {
String[] cmd = {"java","-jar","C:\\<path to selenium>\\server\\selenium-server.jar" };
p = Runtime.getRuntime().exec(cmd);
} catch (IOException e) {
System.out.println("IOException caught: "+e.getMessage());
e.printStackTrace();
}
System.out.println("Waiting for server...");
int sec = 0;
int timeout = 20;
boolean serverReady = false;
try {
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
while (sec < timeout && !serverReady) {
while (input.ready()) {
String line = input.readLine();
System.out.println("From selenium: "+line);
if (line.contains("Started HttpContext[/,/]")) {
serverReady = true;
}
}
Thread.sleep(1000);
++sec;
}
input.close();
} catch (Exception e) {
System.out.println("Exception caught: "+e.getMessage());
}
if (!serverReady) {
throw new RuntimeException("Selenium server not ready");
}
System.out.println("Done waiting");
Some notes on the above code:
Friday, August 29, 2008
SQL INSERTs
The other day, we were confronted with an interesting SQL problem. A user wanted to be able to paste in a set of record IDs to be saved and made available for use throughout the rest of the product. For large sets of data (in the millions of rows), it makes sense for users to upload these files to us via FTP or even through the web server, then have the files loaded into the database. In these cases, users most likely already have the data readily available in some file format, so it’s easy for them to just upload it to us. But in the small to medium size cases (anywhere from hundreds to tens of thousands of records), the user could be pulling the data from an Excel spreadsheet or similar document so that it’s easier for him or her to simply paste the data into a text field of a form. For simplicity’s sake, we can assume that the data consists of a set of ID numbers that are carriage return delimited.
So then, how do we persist these IDs to the database? Essentially we are given a ginormous string of delimited IDs that we want to dump into a table with a single column that is ID number, like so:
In tackling this problem, we considered four different approaches. Our goal was to find the fastest approach (in terms of user wait time), since in many cases the user could be forced to wait several minutes for the upload to complete. As with many enterprise web applications, our DB resides on a different machine than our web server. So each approach’s performance is really driven by two factors:
Query Time – the total amount of time the DB spends processing the queries that insert IDs into the table we designate
NetworkTransfer Time – the total amount of time spent transferring requests from our web server to the DB
The first approach we tried, which we’ll refer to as the Naïve Insert Loop, was to loop through the delimited IDs, inserting a single row into the database for each ID. Each query to insert a row was fired off as a separate DB request:
This approach is problematic for several reasons, all stemming from the fact that it creates an individual query per record ID and fires it off from the web app to the DB one at a time. Since each query and request has a certain overhead to it, this solution pays huge penalties for the large numbers of queries and requests used. We used this naïve approach as a baseline for which to improve upon.
Recognizing that we needed to reduce the number of queries and requests fired, we then considered Improved Insert Loop, which was very similar to the Insert Loop except that we combined the INSERTs together via UNION ALLs before firing them off to the DB:
We can combine these INSERTs together into batches of a thousand* SELECT statements UNION ALLed together, so we essentially have reduced the number of queries and network requests by a factor of thousand. The queries and requests are themselves approximately a thousand times larger than before. But we have improved net performance because by combining a thousand queries together, we don’t have to pay the overhead attached to all of the individual thousand queries and requests we would have run otherwise. For instance, by reducing the number of queries, we are reducing the number of DB transactions, and therefore we reduce the number of disk writes that happen on the DB since we are reducing the number of transaction log flushes.
In both approaches above, we are forced to wrap the IDs in queries that insert them into the table on the DB. If we could somehow transmit the raw IDs to the DB and have them parsed and inserted completely on the DB side, we could greatly reduce the size of the data sent over the network and thus greatly reduce the Network Transfer Time.
With that in mind, we came up with the Stored Procedure Loop approach. Essentially we would pass the entire string of delimited IDs as a TEXT field to a stored procedure, which would do the work of parsing the field and INSERTing the individual records into a target table. Below is the stored procedure definition. It starts by logging the entirety of the paste request into a dataStagingTable and parses the data logged in the table.
This approach yielded a huge performance improvement, as we had essentially minimized the Network Transfer Time by minimizing the amount of data being transmitted. We could have further improved the Query Time by reducing the number of transactions via either explicit transaction blocks or by combining INSERTs into batches as we did with the Improved Loop Insert. But even by making these improvements, we still would have had approximately the same number of queries being run as we did for the Improved Loop Insert (although now they would be run within the stored procedure on the DB side).
In order to further improve the Query Time, we finally arrived at a fourth approach, the BCP Insert. BCP is a utility included with SQL Server that loads data from a file into a DB table. In this case, we dump the delimited IDs into a text file, then invoke BCP on the new text file. To run the BCP utility, we had to make sure there was a way for the web server to transmit these files to the DB machine. After that, we could run the BCP utility as a console command:
Creating the format file for configuring BCP to parse the data correctly was also straightforward:
Like with the Stored Procedure Loop, the data sent is essentially just the raw data, but in this case, SQL loads all the data without having to run a whole slew of queries. Furthermore, SQL’s loading of this data is not logged, which results in much less overhead than the previous approaches. This final approach does require additional time to move data to and from the file system, but in sum it was still faster than the other approaches. It is important to note, however, that as the number of records being pasted decreases, the difference in performance between these approaches also decreases. In fact, if we are inserting fewer than a thousand records, it is actually faster to revert from BCP Insert to one of the more naïve solutions, since the overhead of the file dump begins to dominate the actual Query and Network Transfer Time.
In conclusion, the BCP Insert solution was the fastest of the four. This is not unexpected, since these are the types of operations that BCP was designed to perform quickly. Another interesting build on these solutions would have been to incorporate a means of compression of the raw data before transmitting from web server to DB server. That said, we found this to be an interesting chance to experiment with a few different creative solutions.
* The thousand here is selected for simplicity; we can continue to increase this number to maximize the benefit of these batch combinations. The cap on the batch size is dependent on the RDBMS you are using.
So then, how do we persist these IDs to the database? Essentially we are given a ginormous string of delimited IDs that we want to dump into a table with a single column that is ID number, like so:
CREATE TABLE records
(
recordID INT NOT NULL,
CONSTRAINT PK_records PRIMARY KEY CLUSTERED (recordID)
)
In tackling this problem, we considered four different approaches. Our goal was to find the fastest approach (in terms of user wait time), since in many cases the user could be forced to wait several minutes for the upload to complete. As with many enterprise web applications, our DB resides on a different machine than our web server. So each approach’s performance is really driven by two factors:
The first approach we tried, which we’ll refer to as the Naïve Insert Loop, was to loop through the delimited IDs, inserting a single row into the database for each ID. Each query to insert a row was fired off as a separate DB request:
INSERT INTO records (recordID)
SELECT 12
INSERT INTO records (recordID)
SELECT 15
INSERT INTO records (recordID)
SELECT 17
...
This approach is problematic for several reasons, all stemming from the fact that it creates an individual query per record ID and fires it off from the web app to the DB one at a time. Since each query and request has a certain overhead to it, this solution pays huge penalties for the large numbers of queries and requests used. We used this naïve approach as a baseline for which to improve upon.
Recognizing that we needed to reduce the number of queries and requests fired, we then considered Improved Insert Loop, which was very similar to the Insert Loop except that we combined the INSERTs together via UNION ALLs before firing them off to the DB:
INSERT INTO records (recordID)
SELECT 12
UNION ALL
SELECT 15
UNION ALL
SELECT 17
...
INSERT INTO records (recordID)
SELECT 27
UNION ALL
SELECT 28
UNION ALL
SELECT 54
...
We can combine these INSERTs together into batches of a thousand* SELECT statements UNION ALLed together, so we essentially have reduced the number of queries and network requests by a factor of thousand. The queries and requests are themselves approximately a thousand times larger than before. But we have improved net performance because by combining a thousand queries together, we don’t have to pay the overhead attached to all of the individual thousand queries and requests we would have run otherwise. For instance, by reducing the number of queries, we are reducing the number of DB transactions, and therefore we reduce the number of disk writes that happen on the DB since we are reducing the number of transaction log flushes.
In both approaches above, we are forced to wrap the IDs in queries that insert them into the table on the DB. If we could somehow transmit the raw IDs to the DB and have them parsed and inserted completely on the DB side, we could greatly reduce the size of the data sent over the network and thus greatly reduce the Network Transfer Time.
With that in mind, we came up with the Stored Procedure Loop approach. Essentially we would pass the entire string of delimited IDs as a TEXT field to a stored procedure, which would do the work of parsing the field and INSERTing the individual records into a target table. Below is the stored procedure definition. It starts by logging the entirety of the paste request into a dataStagingTable and parses the data logged in the table.
CREATE TABLE dataStagingTable (
logID int not null identity(1,1),
data Text,
pastingTime datetime
)
exec processPastedData 'wilfred', '2
3
4
5', '
',','
select * from wilfred
CREATE PROCEDURE dbo.processPastedData
@targetTable VARCHAR(32),
@data TEXT,
@rowDelimiter CHAR,
@colDelimiter CHAR
AS
BEGIN
DECLARE @logID INT
DECLARE @dlen BIGINT
DECLARE @offset INT
DECLARE @linePtr INT
DECLARE @buf varchar(4000)
DECLARE @cols varchar(255)
INSERT INTO dataStagingTable (data, pastingTime)
SELECT @data, getDate()
SELECT @logID = @@Identity
SELECT @offset = 1
SELECT @dlen = datalength(data)
FROM dataStagingTable
WHERE logID = @logID
SELECT @cols = 'recordID'
SET NOCOUNT ON
WHILE (@offset > 0)
BEGIN
SELECT @linePtr = CHARINDEX(@rowDelimiter, SUBSTRING(data, @offset, 4000))
FROM dataStagingTable
WHERE logID = @logID
if (@linePtr > 0)
SELECT @buf = REPLACE (SUBSTRING(data, @offset , @linePtr-1), @colDelimiter, ''',''')
FROM dataStagingTable
WHERE logID = @logID
else
SELECT @buf = REPLACE (SUBSTRING(data, @offset , 8000), @colDelimiter, ''',''')
FROM dataStagingTable
WHERE logID = @logID
SELECT @buf = REPLACE (@buf, char(13), '')
EXEC ('INSERT INTO ' + @targetTable + ' (' + @cols + ') SELECT ' + @buf)
SET @offset = @offset + @linePtr
if (@linePtr = 0)
BREAK
END
END
This approach yielded a huge performance improvement, as we had essentially minimized the Network Transfer Time by minimizing the amount of data being transmitted. We could have further improved the Query Time by reducing the number of transactions via either explicit transaction blocks or by combining INSERTs into batches as we did with the Improved Loop Insert. But even by making these improvements, we still would have had approximately the same number of queries being run as we did for the Improved Loop Insert (although now they would be run within the stored procedure on the DB side).
In order to further improve the Query Time, we finally arrived at a fourth approach, the BCP Insert. BCP is a utility included with SQL Server that loads data from a file into a DB table. In this case, we dump the delimited IDs into a text file, then invoke BCP on the new text file. To run the BCP utility, we had to make sure there was a way for the web server to transmit these files to the DB machine. After that, we could run the BCP utility as a console command:
bcp ClientDB.dbo.records in dataDump.txt -S DBMachine -U userID -P password -f formatFile.fmt
------------
ClientDB.dbowner.records = [DATABASE].[SCHEMA].[TABLE TO UPLOAD DATA TO]
dataDump.txt = data file that contains pasted data
-S DBMachine = name of server to connect to
-U userID = tells bcp to use a specific userID to log into DB machine
-P password = tells bcp the password to use with userID to log into DB machine
-f formatFile.fmt = format file that tells bcp how to parse the data file and how to insert into records table
Creating the format file for configuring BCP to parse the data correctly was also straightforward:
8.0
1
1 SQLCHAR 0 50 "\r\n" 1 recordID SQL_Latin1_General_CP1_CI_AS
------------
8.0 = SQL Version
1 = number of columns
Third row going left to right:
1 = File field order
SQLCHAR = Host file data type
0 = Prefix length
50 = Host file data length
"\r\n" = line terminator
1 = Server column order
recordID = name of column we are uploading to
SQL_Latin1_General_CP1_CI_AS = column collation type
Like with the Stored Procedure Loop, the data sent is essentially just the raw data, but in this case, SQL loads all the data without having to run a whole slew of queries. Furthermore, SQL’s loading of this data is not logged, which results in much less overhead than the previous approaches. This final approach does require additional time to move data to and from the file system, but in sum it was still faster than the other approaches. It is important to note, however, that as the number of records being pasted decreases, the difference in performance between these approaches also decreases. In fact, if we are inserting fewer than a thousand records, it is actually faster to revert from BCP Insert to one of the more naïve solutions, since the overhead of the file dump begins to dominate the actual Query and Network Transfer Time.
In conclusion, the BCP Insert solution was the fastest of the four. This is not unexpected, since these are the types of operations that BCP was designed to perform quickly. Another interesting build on these solutions would have been to incorporate a means of compression of the raw data before transmitting from web server to DB server. That said, we found this to be an interesting chance to experiment with a few different creative solutions.
* The thousand here is selected for simplicity; we can continue to increase this number to maximize the benefit of these batch combinations. The cap on the batch size is dependent on the RDBMS you are using.
Thursday, August 14, 2008
Debugging at APT - Part 3: Fiddler
Fiddler has been around for a while, but I was only introduced to it a few months ago. Earlier this year, we were able to budget some time toward improving our software's performance, rather than continuing to add new features as quickly as we had been. Fiddler was especially useful for telling us just how much work our web servers (and our clients' browsers) were doing, and where some of our bottlenecks and improvement opportunities were.
In my first post I mentioned that we have some pretty complex pages throughout our software. Before optimizing them, we were making hundreds of HTTP requests per page (including the now-infamous 85 iframes), and we were applying the same super-strict "no cache" rules to our static files that we applied to our dynamically-generated content. As a result, even our simplest pages required well over 100K of HTTP traffic, and some of the most complex pages were closer to 2MB. Fiddler's biggest strength for us was seeing how each of the various caching directives you can send in HTTP headers affects the browser and network traffic.
Coincidentally, right after I learned about Fiddler, I was asked to help troubleshoot a performance concern raised by one of our newer clients. They were sporadically and unpredictably experiencing "This page cannot be displayed (cannot find server or DNS error)" messages. It was inconsistent but seemed to happen primarily during the client's core business hours, and none of our other clients were experiencing comparable symptoms, so we figured it could be a network problem on their end. A combination of Fiddler (on one client user's computer) and Wireshark (on our web server) showed us a pretty clear pattern of dropped and resubmitted requests at the HTTP level. At the same time, our own internal logging showed an extraordinarily high variance in round-trip performance.
It turned out that the client's internal firewall was sporadically dropping some outgoing requests and incoming responses during times of peak traffic on their general-purpose network. The solution was to establish a VPN tunnel, using a separate infrastructure that they already had in place for their vendors and business partners. In hindsight, we might have reached the same conclusion even more quickly if we had run Fiddler on both sides and compared the traces.
In my first post I mentioned that we have some pretty complex pages throughout our software. Before optimizing them, we were making hundreds of HTTP requests per page (including the now-infamous 85 iframes), and we were applying the same super-strict "no cache" rules to our static files that we applied to our dynamically-generated content. As a result, even our simplest pages required well over 100K of HTTP traffic, and some of the most complex pages were closer to 2MB. Fiddler's biggest strength for us was seeing how each of the various caching directives you can send in HTTP headers affects the browser and network traffic.
Coincidentally, right after I learned about Fiddler, I was asked to help troubleshoot a performance concern raised by one of our newer clients. They were sporadically and unpredictably experiencing "This page cannot be displayed (cannot find server or DNS error)" messages. It was inconsistent but seemed to happen primarily during the client's core business hours, and none of our other clients were experiencing comparable symptoms, so we figured it could be a network problem on their end. A combination of Fiddler (on one client user's computer) and Wireshark (on our web server) showed us a pretty clear pattern of dropped and resubmitted requests at the HTTP level. At the same time, our own internal logging showed an extraordinarily high variance in round-trip performance.
It turned out that the client's internal firewall was sporadically dropping some outgoing requests and incoming responses during times of peak traffic on their general-purpose network. The solution was to establish a VPN tunnel, using a separate infrastructure that they already had in place for their vendors and business partners. In hindsight, we might have reached the same conclusion even more quickly if we had run Fiddler on both sides and compared the traces.
Thursday, July 31, 2008
Debugging at APT - Part 2: IE Developer Toolbar
As I said last time, we don't officially support Firefox yet because our clients use it so rarely. So we were relieved when Microsoft finally started to catch up by releasing the IE Developer Toolbar. We're often frustrated by an inexplicable error (either an actual JS error, or just ugly/broken HTML/CSS) that vexes us in IE but works perfectly fine in Firefox. For the JavaScript errors, we've had the Visual Studio Script Debugger available to us for a while (once we jumped through the necessary hoops to enable it), but sometimes the problem is that some of our JS code just isn't being loaded or executed properly. (Never mind that Script Debugger often shows JS exceptions in the middle of completely unrelated HTML markup.) Since we get some of our JS from asynchronous AJAX calls, it's something that View Source wouldn't be able to tell us. Once again, "select element by click" (equivalent to "Inspect" in Firebug) has been a great friend.
I also like the syntax-highlighted View Source, although I usually end up using something like Notepad++ to view source from a browser. But even better, Developer Toolbar's View Source includes the list of CSS declarations that apply to the portion of the page I've selected.
As you can see above and below, I have selected one element buried pretty deep within one of our pages, and I can easily use "Element Source with Style" to see not only its entire innerHTML but also all the CSS declarations, where they came from, and all parent tags' IDs and styles.
Next time I'll talk about Fiddler, also owned by Microsoft, which helps us debug "over the wire."
I also like the syntax-highlighted View Source, although I usually end up using something like Notepad++ to view source from a browser. But even better, Developer Toolbar's View Source includes the list of CSS declarations that apply to the portion of the page I've selected.
As you can see above and below, I have selected one element buried pretty deep within one of our pages, and I can easily use "Element Source with Style" to see not only its entire innerHTML but also all the CSS declarations, where they came from, and all parent tags' IDs and styles.
Next time I'll talk about Fiddler, also owned by Microsoft, which helps us debug "over the wire."
Friday, July 25, 2008
Javascript Basics: The second and third functions you should write and use all the time
After an embarrassingly long delay, I've decided to put two functions I use all the time in this post to get us somewhat back on track. I also plan on including some other posts about our experimentation with the Yahoo! User Interface (YUI) library and the jQuery library I mentioned last time.
But anyways, back to this post's topic, javascript functions you should write for yourself and be using all the time. This weeks functions are actually pretty simple, but I've talked to and helped a lot of people who weren't using them or had written them so that they're not quite as generic as they should be.
One of the main uses of javascript is to interact with the document object model (DOM) created by the browser. At the most basic level, the ability to take static HTML and manipulate not only the properties of those HTML elements, but also remove those elements as well as create new elements really gives javascript a tremendous amount of power.
Quick digression: Of course, this isn't the only thing javascript does for us. Javascript also plays a key role in asynchronous javascript and XML (AJAX), which has revolutionized the way a lot of users interact with web pages. Some would probably argue that AJAX is even more important than the ability to create dynamic HTML (DHTML). I think they're both critical features, and which you think is more important probably will depend a lot on the web site or application you're developing.
Regardless of whether manipulating the DOM is the most important feature of javascript, everyone can admit it's extremely powerful. One of the most frequent manipulations that developers use javascript for is showing and hiding HTML elements. On pretty much every web page there is some extra or more detailed information that you might not want to display to the user immediately, but would like to reveal when they click a link or check a checkbox. Showing the user only the information they need to know at any particular time can greatly increase the usability of a site.
Here is a simple function that will automatically display an element based on its ID:
Notice that this function builds off of the $() function from the first post that will get an HTML element based on its ID. It gets a reference to that element, accesses the style attribute of that element, and finally sets the display property of the style attribute to the empty string. At this point you might have a couple questions worth addressing.
First, it's worth pointing out that the style attribute of an element is not a simple value as it is for an element's ID or class. In this case, the style attribute is an object because there are a lot of styles that might apply to a particular element such as the font size, positioning on the page, or background color. All of these style properties could be stored as one big string, but it's pretty easy to see how that would be a nightmare for developers (editing an element's "font-size" wouldn't be very easy).
The other two questions worth answering are why did we choose to change the display property and why did we set it to the empty string? These are both good questions. There are two style properties that control whether an item can be seen or not, display and visibility. The visibility property can have a value of "hidden". It might seem like this would do what we want, but what the visibility property actually does is changes whether an element is visible without removing the space it takes up from the page. This means if a I have a paragraph element whose visibility I set to hidden, I won't be able to see that paragraph element, but I will see a big white space where that element should be. This is rarely what you want, but sometimes it is. Usually what you want is for that element to take up space on the page when it is visible or displayed, but be removed from the flow or layout of the page when it is not being displayed. This is exactly what the display property does. When the display property for an element is set to "none", then that element is completely removed from the flow of the page so there is no unnecessary white space.
So the only question remaining is why did we set the display property to the empty string in order to get our element to appear. If anything, an empty string might indicate that something shouldn't be displayed. In this case, as is the case with a lot of the style properties, it is set to the default value for that property. It just so happens that the default value for the display property is whatever display value for that element will display it properly. This means for block elements like <div> or <p> elements, the display value will be set to "block". For inline elements such as <strong> or <span> elements, it will be "inline". Certain elements, such as elements, even have different display types in different browsers. In Internet Explorer, a <tr> is displayed properly with the value of "block", but needs to be set to "table-row" in Firefox. This is where setting the display value to the empty string and allowing the browser to then apply the default really shines.
So at this point it should be pretty clear why this function was written this way (but if it isn't, let me know in the comments). It's nice to have all of these somewhat complex choices encapsulated in a single easy to understand function that anyone can use, not to mention there's a lot less typing to do.
So what is the other function for this post?
I don't think this one needs much explanation, so here it is:
I don't think there's anything I need to say about this function. The only slightly interesting thing to note here is that, unlike displaying elements, if you don't want display elements, setting their display value to "none" always does the trick.
Cool, so that's it for this post. Add these functions to wherever you put the $() function -- hopefully an external javascript file that you include on pages you need javascript on. Then use them in your javascript and see how much shorter and easier to understand your javascript becomes!
The Motivation:
The Problem:
The Solution:
ColdFusion’s default reporting mechanism for uncaught errors is to dump the error information to the console. This has several disadvantages, two of them being that the console would have to be monitored and whoever is monitoring it is limited to the dumped information. To address this issue, ColdFusion Administrator has a setting for a Site-wide Error Handler. This is a custom template to execute when an error is encountered, and allows for sophisticated post-processing of the error.
At APT, we take advantage of this feature and have a very robust Site-Wide Error Handler. Among other things, it retrieves additional information about the user and setup, retrieves information about the product’s state, diagnoses the error, logs the error to a database, and sends an e-mail to relevant engineering and delivery team members. But there is a catch: if an error is encountered within the Site-wide Error Handler it stops processing and dumps that error to the console. Should this happen, we are right back where we started.
The solution is simple: make sure your Site-wide Error Handler never fails. However, if you have a complicated template, this is easier said than done. In this situation, we must make the Site-wide Error Handler fault-tolerant.
How do we achieve fault-tolerance? The foundation is liberal use of try-catch blocks within your Site-wide Error Handler. While it may at first appear sloppy and clutter up the code, it is critical for isolating the effects of errors, making sure the maximum amount of processing takes place, and ensuring the most information possible makes it back to you.
Look at this simple example template:
If retrieving user information fails, we will still be sent an e-mail containing the error information and information about the product’s state. This is exactly the isolation we are going for. Subcomponents within in the try-catch blocks can be wrapped in their own try-catch blocks as well, creating an even more granular and fault-tolerant template.
When errors with your Site-wide Error Hander are caught, be sure you attempt to report information on them instead of letting them get swallowed up silently. In the above example, the information about the error encountered while retrieving user information should be included in the final e-mail sent. This feedback is critical in debugging and improving the Site-wide Error Handler—you will be glad you have it. However, be sure to wrap this reporting in its own try-catch in case there is an error with reporting the error. The overall theme is: the more paranoid the better!
By treating your Site-wide Error Handler as a series of granular tasks contained within try-catch blocks, the effect of failures will be limited while the maximum amount of error information safely makes its way back to you. This will be speed up the time it takes to identify, diagnose, and fix bugs and thus improve the quality of your software. <cftry> and <cfcatch> tags are essentially free, but engineering time is not.
But anyways, back to this post's topic, javascript functions you should write for yourself and be using all the time. This weeks functions are actually pretty simple, but I've talked to and helped a lot of people who weren't using them or had written them so that they're not quite as generic as they should be.
2. show()
One of the main uses of javascript is to interact with the document object model (DOM) created by the browser. At the most basic level, the ability to take static HTML and manipulate not only the properties of those HTML elements, but also remove those elements as well as create new elements really gives javascript a tremendous amount of power.
Quick digression: Of course, this isn't the only thing javascript does for us. Javascript also plays a key role in asynchronous javascript and XML (AJAX), which has revolutionized the way a lot of users interact with web pages. Some would probably argue that AJAX is even more important than the ability to create dynamic HTML (DHTML). I think they're both critical features, and which you think is more important probably will depend a lot on the web site or application you're developing.
Regardless of whether manipulating the DOM is the most important feature of javascript, everyone can admit it's extremely powerful. One of the most frequent manipulations that developers use javascript for is showing and hiding HTML elements. On pretty much every web page there is some extra or more detailed information that you might not want to display to the user immediately, but would like to reveal when they click a link or check a checkbox. Showing the user only the information they need to know at any particular time can greatly increase the usability of a site.
Here is a simple function that will automatically display an element based on its ID:
function show(id) {
$(id).style.display = "";
}
Notice that this function builds off of the $() function from the first post that will get an HTML element based on its ID. It gets a reference to that element, accesses the style attribute of that element, and finally sets the display property of the style attribute to the empty string. At this point you might have a couple questions worth addressing.
First, it's worth pointing out that the style attribute of an element is not a simple value as it is for an element's ID or class. In this case, the style attribute is an object because there are a lot of styles that might apply to a particular element such as the font size, positioning on the page, or background color. All of these style properties could be stored as one big string, but it's pretty easy to see how that would be a nightmare for developers (editing an element's "font-size" wouldn't be very easy).
The other two questions worth answering are why did we choose to change the display property and why did we set it to the empty string? These are both good questions. There are two style properties that control whether an item can be seen or not, display and visibility. The visibility property can have a value of "hidden". It might seem like this would do what we want, but what the visibility property actually does is changes whether an element is visible without removing the space it takes up from the page. This means if a I have a paragraph element whose visibility I set to hidden, I won't be able to see that paragraph element, but I will see a big white space where that element should be. This is rarely what you want, but sometimes it is. Usually what you want is for that element to take up space on the page when it is visible or displayed, but be removed from the flow or layout of the page when it is not being displayed. This is exactly what the display property does. When the display property for an element is set to "none", then that element is completely removed from the flow of the page so there is no unnecessary white space.
So the only question remaining is why did we set the display property to the empty string in order to get our element to appear. If anything, an empty string might indicate that something shouldn't be displayed. In this case, as is the case with a lot of the style properties, it is set to the default value for that property. It just so happens that the default value for the display property is whatever display value for that element will display it properly. This means for block elements like <div> or <p> elements, the display value will be set to "block". For inline elements such as <strong> or <span> elements, it will be "inline". Certain elements, such as
So at this point it should be pretty clear why this function was written this way (but if it isn't, let me know in the comments). It's nice to have all of these somewhat complex choices encapsulated in a single easy to understand function that anyone can use, not to mention there's a lot less typing to do.
So what is the other function for this post?
3. hide()
I don't think this one needs much explanation, so here it is:
function hide(id) {
$(id).style.display = "none";
}
I don't think there's anything I need to say about this function. The only slightly interesting thing to note here is that, unlike displaying elements, if you don't want display elements, setting their display value to "none" always does the trick.
Cool, so that's it for this post. Add these functions to wherever you put the $() function -- hopefully an external javascript file that you include on pages you need javascript on. Then use them in your javascript and see how much shorter and easier to understand your javascript becomes!
Monday, July 21, 2008
Debugging at APT - Part 1: Firebug
Like most developers (using any language or platform), folks at APT crave robust and powerful debugging tools. Fortunately, ColdFusion 8 introduced some fantastic new debugging and monitoring tools, on top of the robust logging and request debugging that has been available for a long time. Unfortunately, our architecture seems to prevent us from using most of those tools. (But we'll have plenty of architecture discussions later.) More generally, our system is not "pure" ColdFusion -- we also use Java directly, as well as SQL, JavaScript, CSS, .NET, VBScript, Ant... you name it -- so we often wonder which debugging tools are best for which jobs.
I recently hosted an internal "Wednesday Noon Session" on debugging at APT. (Every three weeks or so, one of our engineers puts together a presentation that focuses on one aspect of development at APT.) I learned a lot by preparing my session, so I'm going to post a series of articles on the tools we use and how we use them. I'll start today with Firebug.
Firebug
For the client side, as well as some server-side debugging, Firebug is one of our best friends. We don’t officially support Firefox for our clients, because their IT departments pretty much all mandate IE6 and prohibit the installation of any other browser. In fact, 95% of our product logins come from IE6, and almost all of the rest are from IE7. Once in a while we even see someone try to use IE6 on Win2K, which typically is missing the past five years worth of bug fixes and security patches. (Yes, even in mid-2008!)
But we’re working toward full Firefox support for a number of reasons, one of which is debugging capability. We have all sorts of complex pages in our software. If you’re not familiar with them, simply finding an element on the page, let alone debugging a script, can be daunting. Thank goodness for the "Inspect" button.
We make heavy use of the Script tab to evaluate JS expressions on-the-fly and step through functions, as well as the Net tab to see what's going on with our AJAX requests (like sorting, searching and pagination within each component). We also used Yahoo!'s YSlow extension to discover that our menus were rendering upwards of 85 iframe tags on each page. (Ouch! Well, it turns out that all of the iframes were there to work around a well-known IE6 bug. But we've since gotten smarter about the "hidden iframe hack" and we've also switched to using YUI menus, which are also smarter about using iframes and about rendering efficiency in general.)
Next time: Firebug's cousin and competitor, the IE Developer Toolbar.
I recently hosted an internal "Wednesday Noon Session" on debugging at APT. (Every three weeks or so, one of our engineers puts together a presentation that focuses on one aspect of development at APT.) I learned a lot by preparing my session, so I'm going to post a series of articles on the tools we use and how we use them. I'll start today with Firebug.
Firebug
For the client side, as well as some server-side debugging, Firebug is one of our best friends. We don’t officially support Firefox for our clients, because their IT departments pretty much all mandate IE6 and prohibit the installation of any other browser. In fact, 95% of our product logins come from IE6, and almost all of the rest are from IE7. Once in a while we even see someone try to use IE6 on Win2K, which typically is missing the past five years worth of bug fixes and security patches. (Yes, even in mid-2008!)
But we’re working toward full Firefox support for a number of reasons, one of which is debugging capability. We have all sorts of complex pages in our software. If you’re not familiar with them, simply finding an element on the page, let alone debugging a script, can be daunting. Thank goodness for the "Inspect" button.
We make heavy use of the Script tab to evaluate JS expressions on-the-fly and step through functions, as well as the Net tab to see what's going on with our AJAX requests (like sorting, searching and pagination within each component). We also used Yahoo!'s YSlow extension to discover that our menus were rendering upwards of 85 iframe tags on each page. (Ouch! Well, it turns out that all of the iframes were there to work around a well-known IE6 bug. But we've since gotten smarter about the "hidden iframe hack" and we've also switched to using YUI menus, which are also smarter about using iframes and about rendering efficiency in general.)
Next time: Firebug's cousin and competitor, the IE Developer Toolbar.
Friday, June 6, 2008
Javascript Basics: Five functions you should write and use all the time
If you're new to Javascript, you've either already found out or will very shortly that there are some tasks that you need to do all the time. It can get pretty tedious having to remember what the code is for those tasks, or even worse, knowing what the code is, but still having to type it out each time you want to use it.
In this article, I'm going to walk through the first of five functions I use frequently when writing Javascript code that will make not only make writing code easier because the code is more concise, but also because in a lot of cases it's easier to understand. Certainly this series won’t cover all of the shortcut functions you may want to write, but if you're new to Javascript this should get you going so that soon you can write your own. So let's get started:
If you've written Javascript for more than 15 minutes, you've probably used the “document.getElementById” function. If you've written Javascript for more than 30 minutes, you've probably made the mistake of capitalizing the last "d" and had a Javascript error thrown. Hopefuly the error message from the browser made it obvious what was wrong, but in some browsers deciphering the error message can be an art in itself. Because it is easy to misspell and requires typing out 24 characters to accomplish one of the most basic and frequent tasks in Javascript, it doesn’t take long to get frustrated with the “document.getElementById” function.
If you've ever used the popular jQuery library, you know you can get a reference to an HTML element by writing:
The folks who wrote jQuery felt your pain in writing out "document.getElementById" each time, and made a shortcut function whose name was just the dollar sign. Because the dollar sign is a valid character for use in a Javascript function name but it's not a character that is often used, they chose to create a function that was only one character long to effectively act as an alias for the "document.getElementById" function.
This was such a great idea, we’ve copied it for ourselves at APT and included it in our own Javascript library by writing the following function: (Note: If you use jQuery already, there’s no need to write this function yourself since you already have it!)
Armed with this code, we can now replace all instances of "document.getElementById" with the much shorter "$".
Another side benefit of using this function that hasn’t been mentioned yet is that every time we get a reference to an element, we've reduced the size of our code by over 93% (well, not exactly, but you get the idea). If you add the space saved across a large Javascript file by making this replacement, the result is usually pretty significant.
You'll also notice from our implementation, that when we call our “$’ function, we’re not just directly calling the “document.getElementById” function. Instead we built a little bit of extra functionality into our “$” function.
Notice that if you pass in an object that is not a string, our function won't throw an error, it will just pass back the same object that was passed in. This can be useful in functions where you want your code to be flexible for whoever is using it so that they can pass in either a string that is the ID of an HTML element or a reference to the HTML element itself. At the top of your code you can pass that object through our “$” function, and rest assured that what is returned is a reference to a HTML element.
As a final point, some people don't like using the dollar sign as the function name. You can feel free to name this function whatever you want. For instance, you could name it "id" instead. I prefer using "$" because I can imagine writing code where I name a variable "id" and accidentally overwrite the function I created, which could then lead to all sorts of funky errors. If I use "$", the chances of that happening are greatly reduced.
Alright, so one down. I'll post the next two functions tomorrow which are related to showing and hiding HTML elements.
In this article, I'm going to walk through the first of five functions I use frequently when writing Javascript code that will make not only make writing code easier because the code is more concise, but also because in a lot of cases it's easier to understand. Certainly this series won’t cover all of the shortcut functions you may want to write, but if you're new to Javascript this should get you going so that soon you can write your own. So let's get started:
1. $() or id()
If you've written Javascript for more than 15 minutes, you've probably used the “document.getElementById” function. If you've written Javascript for more than 30 minutes, you've probably made the mistake of capitalizing the last "d" and had a Javascript error thrown. Hopefuly the error message from the browser made it obvious what was wrong, but in some browsers deciphering the error message can be an art in itself. Because it is easy to misspell and requires typing out 24 characters to accomplish one of the most basic and frequent tasks in Javascript, it doesn’t take long to get frustrated with the “document.getElementById” function.
If you've ever used the popular jQuery library, you know you can get a reference to an HTML element by writing:
var referenceToElement = $("elementID");
The folks who wrote jQuery felt your pain in writing out "document.getElementById" each time, and made a shortcut function whose name was just the dollar sign. Because the dollar sign is a valid character for use in a Javascript function name but it's not a character that is often used, they chose to create a function that was only one character long to effectively act as an alias for the "document.getElementById" function.
This was such a great idea, we’ve copied it for ourselves at APT and included it in our own Javascript library by writing the following function: (Note: If you use jQuery already, there’s no need to write this function yourself since you already have it!)
$ = function(id) {
// If "id" is a string, return the value of "document.getElementById",
// otherwise, just return whatever was passed in.
return (typeof id == "string") ? document.getElementById(id) : id;
}
Armed with this code, we can now replace all instances of "document.getElementById" with the much shorter "$".
Another side benefit of using this function that hasn’t been mentioned yet is that every time we get a reference to an element, we've reduced the size of our code by over 93% (well, not exactly, but you get the idea). If you add the space saved across a large Javascript file by making this replacement, the result is usually pretty significant.
You'll also notice from our implementation, that when we call our “$’ function, we’re not just directly calling the “document.getElementById” function. Instead we built a little bit of extra functionality into our “$” function.
Notice that if you pass in an object that is not a string, our function won't throw an error, it will just pass back the same object that was passed in. This can be useful in functions where you want your code to be flexible for whoever is using it so that they can pass in either a string that is the ID of an HTML element or a reference to the HTML element itself. At the top of your code you can pass that object through our “$” function, and rest assured that what is returned is a reference to a HTML element.
function myCoolFunction(eitherAnIdOrAnElementReference) {
var definitelyAnElementReference = $(eitherAnIdOrAnElementReference);
// Do something to that element...
}
As a final point, some people don't like using the dollar sign as the function name. You can feel free to name this function whatever you want. For instance, you could name it "id" instead. I prefer using "$" because I can imagine writing code where I name a variable "id" and accidentally overwrite the function I created, which could then lead to all sorts of funky errors. If I use "$", the chances of that happening are greatly reduced.
Alright, so one down. I'll post the next two functions tomorrow which are related to showing and hiding HTML elements.
Wednesday, May 21, 2008
VBScript Syntax Checking
This post describes how to use VBScript to check the syntax of a given block of VBScript code.
The Motivation:
At APT, one software suite we use to run automated GUI tests of our web-based software is Mercury QuickTest Professional. QuickTest runs tests written in VBScript that click through a prescribed path in our software to make sure we don't throw any errors.
Rather than store hundreds of static .vbs files containing the definition of tests, we store our test definitions in a SQL database and dynamically generate the corresponding .vbs files on demand. We do this for a few reasons:
- It makes test maintenance easier (you maintain only the structure of the test, not the code's syntax)
- It enables users who are unfamiliar with VBS to author and maintain tests
- By introducing separation between the test's definition and execution, we can change how we execute our test without changing our test's definition (for example, migrating test execution to another platform such as Selenium, which can execute different languages)
The Problem:
One of the challenges of dynamically generating code is making sure that what you generate is syntactically valid. Making the assumption that your generated code will be perfect is generally a bad idea, because:
- Your code-generation code may contain errors
- As you modify your test definition model, the code-generation code can fall out of sync, resulting in invalid generated code
- You'll likely want to support user-entered "custom" sections of code, which you can't assume will be syntactically valid
The Solution:
After we generate the VBS code for the requested test, we check its syntax prior to executing it by calling the function defined below:
Public Function checkVBSSyntax(vbsCode)
' Save any pre-existing error so we can revert to it
oldErr = Err
Err.Number = 0
On Error Resume Next
' Execute the vbs code to see if it throws an error
ExecuteGlobal vbsCode
hasError = Err.Number <> 0
On Error Goto 0
' Revert to the pre-existing error
Err = oldErr
checkVBSSyntax = (Not hasError)
End Function
What we're doing here is simply executing the questionable code in the global scope to see if it throws an error. If it throws an error, we assert that it must be due to a syntax error.
A couple of notes:
- This function assumes that the questionable code contains only Function and Sub definitions; that is, no code exists outside of a Function or Sub declaration. If it does contain code outside of a Function or Sub declaration, that code will actually be executed (not just checked for its syntax). If you have such code, wrap it in a Public Sub main() ... End Sub statement.
- This has the side effect of defining all the Functions / Subs in your questionable code in global scope; that is, after calling this function, you'll be able to call any functions defined in your questionable code, as if you included it as a function library. As a result, you'll want to be careful that Function / Sub names in the questionable code don't collide with Function / Sub names in your syntax checking code.
Friday, May 2, 2008
Fault-Tolerant ColdFusion Error Reporting
ColdFusion’s default reporting mechanism for uncaught errors is to dump the error information to the console. This has several disadvantages, two of them being that the console would have to be monitored and whoever is monitoring it is limited to the dumped information. To address this issue, ColdFusion Administrator has a setting for a Site-wide Error Handler. This is a custom template to execute when an error is encountered, and allows for sophisticated post-processing of the error.
At APT, we take advantage of this feature and have a very robust Site-Wide Error Handler. Among other things, it retrieves additional information about the user and setup, retrieves information about the product’s state, diagnoses the error, logs the error to a database, and sends an e-mail to relevant engineering and delivery team members. But there is a catch: if an error is encountered within the Site-wide Error Handler it stops processing and dumps that error to the console. Should this happen, we are right back where we started.
The solution is simple: make sure your Site-wide Error Handler never fails. However, if you have a complicated template, this is easier said than done. In this situation, we must make the Site-wide Error Handler fault-tolerant.
How do we achieve fault-tolerance? The foundation is liberal use of try-catch blocks within your Site-wide Error Handler. While it may at first appear sloppy and clutter up the code, it is critical for isolating the effects of errors, making sure the maximum amount of processing takes place, and ensuring the most information possible makes it back to you.
Look at this simple example template:
<cftry>
Retrieve user information
<cfcatch> Handle exception </cfcatch>
</cftry>
<cftry>
Retrieve information about the product’s state
<cfcatch> Handle exception </cfcatch>
</cftry>
<cftry>
Send an e-mail with error and additional
retrieved information
<cfcatch> Handle exception </cfcatch>
</cftry>
If retrieving user information fails, we will still be sent an e-mail containing the error information and information about the product’s state. This is exactly the isolation we are going for. Subcomponents within in the try-catch blocks can be wrapped in their own try-catch blocks as well, creating an even more granular and fault-tolerant template.
When errors with your Site-wide Error Hander are caught, be sure you attempt to report information on them instead of letting them get swallowed up silently. In the above example, the information about the error encountered while retrieving user information should be included in the final e-mail sent. This feedback is critical in debugging and improving the Site-wide Error Handler—you will be glad you have it. However, be sure to wrap this reporting in its own try-catch in case there is an error with reporting the error. The overall theme is: the more paranoid the better!
By treating your Site-wide Error Handler as a series of granular tasks contained within try-catch blocks, the effect of failures will be limited while the maximum amount of error information safely makes its way back to you. This will be speed up the time it takes to identify, diagnose, and fix bugs and thus improve the quality of your software. <cftry> and <cfcatch> tags are essentially free, but engineering time is not.
Wednesday, April 30, 2008
What is "An APT Developer?"
Before we get into why we are all here, we'd like to start by saying, "Welcome"! We hope you get as much out of reading these materials as we have by putting them together!
So, "What is 'An Apt Developer'?" you ask. Quite simply, it is a brain dump of all things software happening at Applied Predictive Technologies or APT for short ("An Apt Developer" - clever, isn't it?). APT is a small, growing software company located in Arlington, Virginia that provides its customers with cutting edge business analytics delivered through everyone's favorite medium - the internet! Read more about us here: http://www.predictivetechnologies.com.
Delivering an intuitive, yet rich set of analytic tools, driven by volumes of performance data, via the web has forced us to overcome numerous engineering challenges. Certainly a large number of these are themed around integrating the many technologies we use (Java, ColdFusion, SQL Server, .Net, AJAX, CSS, XHTML, etc.), though not all are technical in nature. As we have grown, we have also learned much about QA, testing, and product management processes. We hope this blog will reflect on much of what we have learned and serve as a way to give back to the broad software community that has already helped us in many ways. Enjoy!
So, "What is 'An Apt Developer'?" you ask. Quite simply, it is a brain dump of all things software happening at Applied Predictive Technologies or APT for short ("An Apt Developer" - clever, isn't it?). APT is a small, growing software company located in Arlington, Virginia that provides its customers with cutting edge business analytics delivered through everyone's favorite medium - the internet! Read more about us here: http://www.predictivetechnologies.com.
Delivering an intuitive, yet rich set of analytic tools, driven by volumes of performance data, via the web has forced us to overcome numerous engineering challenges. Certainly a large number of these are themed around integrating the many technologies we use (Java, ColdFusion, SQL Server, .Net, AJAX, CSS, XHTML, etc.), though not all are technical in nature. As we have grown, we have also learned much about QA, testing, and product management processes. We hope this blog will reflect on much of what we have learned and serve as a way to give back to the broad software community that has already helped us in many ways. Enjoy!
Subscribe to:
Posts (Atom)