Wednesday, January 12, 2005

The XmlHTTPRequest function was designed so that the client can get small pieces of information from the server without having to do a page refresh. Obviously, the developers thought that XML will be the format of choice for this data exchange.

Thank God they didn’t make XML validation necessary!

I say this because XML will not be the format of choice for this data exchange. Let’s see why.

Firstly, let’s see how the data flow will be in the case of XML over XmlHTTPRequest. A typical scenario is where the data lies in a database on the server, and the data is requested by the client. The steps in the process will be:

Server reads the request, extracts the data from the database.

Server prepares an XML document using this data.

This XML is sent to the client.

Client validates the XML for well-formed-ness.

Client extracts data from the XML markup.

Client calls functions/triggers events which use this data as parameters in some way or other.

For several reasons, this model is not optimum. The questions that come to the mind immediately are:

Is it necessary to prepare an XML document (step 2 above)? Isn’t that an extra step that is probably not necessary?

Client validation is necessary (step 4 above) to ensure that the XML parser can parse the XML. This puts heavy processing requirements on the client. Is this really necessary?

The data from the XML will then have to be extracted (step 5 above). This isn’t really a necessary step either if the data is not marked up as XML. So, then, is it really necessary to use XML?

Then there are more problems. In web applications, where XmlHTTPRequest is most likely to be used, speed is of utmost importance, and optimizing for fast response times will be imperative. In such cases, the amount of time taken for any kind of operation, especially if it happens over the network or the internet, has to be minutely scrutinized and optimized. In such cases, XML is probably not the best suited format for data exchange. XML files are heavy as far as sizes go. A simple text like “Hello World” (11 characters) can be transmitted over XmlHTTPRequest as 11 bytes of data. Marking this up as XML will increase this at least 4 fold (say, and that’s still a conservative estimate) even if whitespaces are stripped off the document, increasing download sizes four times, and reducing response by a factor of 4. Point is, XML comes with its overheads which cannot be tolerated in a web application.

So, in what format will data be transferred over XmlHTTPRequest? The simplest way is as a plain old string of data. This will be the most efficient as far as download speeds are concerned. You will download only 11 bytes where 11 bytes of download is required, not more. Besides, there won’t be any XML parsing or validation in the picture, making the application that much faster.

However, somehow, I don’t see data flowing over XmlHTTPRequest in the form of plain strings of data either. Instead, I think it’ll flow in the form of JavaScript code.

Let me use an example from the guy who deconstructed Google Suggest, and the way XmlHTTPRequest works there. According to him, the code returned by Google when you hit a keystroke looks like this.

Basically, what this code does is call a function, with a bunch of parameters. The function called, sendRPCDone, is defined in client side JavaScript code. The parameters are generated depending on server side processing (probably from Google’s cached searches). Then, later in the code, this response text is run through an eval function to execute the function on the client side (eval(_xmlHttp.responseText);).

I think the eval function is really a boon here. It makes JavaScript as the language of choice over XML for XmlHTTPRequest. For who don’t know what eval does, it basically takes a piece of string, treats it as though it is JavaScript code, and executes it. The string could be dynamically generated by client-side code, or as in this case, returned from the server. Just what we need here!

To summarize, here’s why we’ll use JavaScript instead of XML when using XmlHTTPRequest.

Downloading XML data will take more time as compared to JavaScript. Depending on the amount of data to be transferred, this difference might be huge.

Use of XML will necessitate validation of the markup, so that a parser can read the document – a process that requires extra processing on the client-side. This is not necessary with JavaScript.

The XML data is not already in a position to call scripts. Client side code will have to handle that, based on event handlers or such. JavaScript, on the other hand will be ready to be executed on the client.

It's easy to see why XML will not be used as the format of choice over XmlHTTPRequest. A little thought later, JavaScript emerges as the winner as the format of choice for data exchange over XmlHTTPRequest.

Did someone say that JavaScript will be coming out of the closet this year?

It suits our needs. It's not bulletproof, but I think it could be with minimal effort.

It should work with any W3C-DOM browser.

While I've no accurate means by which to benchmark this, I imagine it would be more responsive than XMLHttp+eval method. (eval() is JavaScripts most performance deficient function).

I'm not saying this is a replacement for XMLHttpRequest. In fact, I am also using XMLHttpRequest (More accurately: Msxml2.XMLHTTP) at work. I use it in a server-side table sort routine. When a sortable table column contains too much data too sort on the client, I query the application framework to do the job (it has the data in session already) and it returns the data as XHTML.

I'm just saying, for the popular purpose of XMLHttp that I see being discussed (and used) here and elsewhere, it is nothing extraordinary.

>>It should work with any W3C-DOM browser.-not Safari. Not old Opera either. Maybe newest, forget offhand. I've used it for ages to send items to the shopping cart when ordering and works pretty well. (Btw, your code needs a timestamp added so not cached by IE. Also IE Mac needs a different routine; you'll crash it with that code.)

>>(eval() is JavaScripts most performance deficient function).-why not just send the data string and and call the function with it? No eval needed.

>> why not just send the data string and and call the function with it? No eval needed.

How would that work? Let's take the code in question in my post. The server returns the string which is basically that sendRPCDone function call. When that is received by the client as a result of the XmlHTTPRequest response, it'll be for all practical purposes just a piece of string. How do you get it to be treated as a piece of client-side code and execute it, without using eval?

The next obvious question would be, is it better to eval a string, or better to parse a string and decide what client-side function to call? Which one is more efficient? Which one is faster? Which one would reduce memory leaks (which is increasingly becoming an issue with JS web apps)?

I'd think that for small scripts that essentially have very few callbacks, your technique would reign supreme. However for larger scripts with more callbacks, parsing the response string to determine the operation would be just as performance intensive as using eval, what with complex if-else ladders.

Of course, that's off the top. I have not tested it, nor could I come across anyone who has. Can anyone shed some light on this matter?

The idea of outputting lots of javascript directly from server side code is a bit of a false economy due to changes in javascript requiring rebuilding solutions. Combining with XSLT on the server side can give the flexibility to change without having to rebuild solutions, And also means you can output HTML directly.

Just want to say, most helpful article for me. I've been struggling to get decent performance for a week, bashing my XML-reading code this way and that. Switching to eval was a perfect solution for getting thousands of records into my Javascript in a tolerable timeframe. Magic.

This post assumes that JavaScript is available, which is around 96% of the typical Internet audience.

In case you are concerned about cases where users do not have JavaScript enabled, you should find out more about "Unobtrusive JavaScript" and "Graceful Degradation" techniques. It is a little harder, but not impossible, to cater to users that don't have JavaScript enabled.