This is a plea for all Ajax developers to carefully consider the MIME Type of all those little snippets of data that are served up for processing by your javascript. If you use an inappropriate Content-Type in your HTTP response, you may find that your carefully crafted applications break when used from mobile phones, or for dial-up users trying to boost their bandwidth using a "web accelerator".

Sure, it's not likely to happen, but it could happen to you. So it probably will. Here I explain the problem to be aware of, and a simple solution to ensure your applications don't succumb to it.

Content-Type pre-Ajax

In the traditional synchronous web model, it is the browser itself that processes the data retrieved from the web server. The browser uses the HTTP Content-Type header supplied by the server to determine how to display the data contained in the body of the response. For example, a GIF image is normally served using a Content-Type of img/gif. When the same data is served as text/html it is dutifully displayed as (gibberish) text.

Aside: Ok if you're an IE user you will have noticed that it ignores the Content-Type and displays the image even when it's served as text/plain. As Alan Flavell has exhaustively documented, this is because IE is wrong and not obeying a mandatory requirement of the HTTP specification. Try the above links in a conforming browser like Firefox and you'll see the correct behaviour. For the purposes of this discussion IE's odd behaviour does not matter.

Content-Type with Ajax

In the Ajax web model, it is your javascript code that processes the data received from the server, not the browser itself. So you can use any content-type you like, right? After all, it's your web server, you know what the content will be and you are in control of how it is processed, regardless of the content type. Except of course, that this isn't always true.

Web Accelerators and Content Optimisation

Consider the road warrior, with Firefox on their laptop and a GPRS connection, using their favourite Ajax-enabled website, hopefully yours. Here in the UK, the network operators are trying to make more effective use of the limited bandwidth available with the current technologies. This can be done by optimising the content received from the web server before it is transmitted over the relatively slow part of the link from the network provider to the mobile handset.

One very simple method of reducing the size of a webpage is to strip redundant information out of the content, such as extraneous whitespace or comments. Indeed, there are many tools available to the Ajax developer to perform precisely this kind of optimimsation before the content is even served, such as the Dojo compressor.

In the case of the mobile network, the optimisation is done on the fly by a gateway that intercepts all HTTP traffic. The gateway relies on the Content-Type reported by the server so it can strip irrelevant data from the content without affecting it's meaning. For example, extraneous whitespace can be safely stripped from HTML (text/html) and the page will render the same, but removing whitespace formatting from a text document that is intended to be read by a human (text/plain) would not be terribly helpful.

Similar transformations can be applied to CSS and javascript. Even images can be automatically "optimised" by reducing their bitdepth and/or quality.

Content Optimistation and Ajax

What does the existence of such on-the-fly optimisers mean for Ajax developers?

As we have seen, HTML may be altered by a helpful optimiser. This means that Ajax apps can break subtly if they request and process HTML documents in certain ways. Consider:

new Ajax.Request(
  '/foo/bar', {
    onSuccess: function(t) {
      var pieces = t.responseText.split('<!--break-->');
      Element.update($('firstdiv'), pieces[0]);
      Element.update($('seconddiv'), pieces[1]);
    }
  }
);

This is a simplified version of some real code I observed in the wild. The intention is to use Prototype's Ajax.Request to asyncronously update the content of two separate divs on the page. The content returned is two fragments of HTML separated by a <!--break-->. The comment acts as a separator with which the javascript splits out the two separate HTML fragments to update the two separate divs.

But '/foo/bar' is served up as text/html so when it passes through an on-the-fly optimiser, the comment gets stripped out because it is considered to be semantically unnecessary. And so the javascript puts both HTML fragments into the firstdiv, probably not what you or your user expected. You can see the resulting problem in action on this test page.

The fix

Whether you agree with the methods employed by such optimisers or not, they do exist and are actively used on low bandwidth mobile networks. But once you know that such alterations to your content can happen, it is easy to avoid any problems it could cause; your javascript application should treat all content in semantically the same way that a browser would when faced with the same Content-Type.

For example, in the above scenario, the comments in the HTML were semantically important to the javascript. The optimiser had assumed that it was delivering HTML content to a web browser which does not consider comments semantically important. There are two ways to fix the above example:

1. Change the javascript - instead of splitting the HTML into chunks using a comment as a divider, split on something semantically important like <div id="break"></div>. This shouldn't get stripped from the HTML when it is optimised.
2. Change the served data - instead of serving the snippets as text/html, use a Content-Type where the comments are preserved e.g. text/plain. Javascript doesn't care what the Content-Type is - it's up to your application code to process the data as it sees fit.

The quick fix

If you don't have time to think about the correct Content-Type for all your Ajax snippets, a quick fix is to use text/plain for all Ajax snippets. Of course, then your application won't benefit from the text/html optimisation as it passes through the optimiser - but your HTML is already optimised for size...

Isn't it?