Tag Archives: XMLHttpRequest

I’ve been dealing with the situation when your server redirects after a DELETE. Which is actually a fairly common use case. User deletes a blog post, redirect them to the index of all posts. The article below has some Rails workarounds, but is relevant to anyone using XHR and perhaps HTML5 History, ie many single-page web apps.

It’s important stuff to know if you’re HTML5ing in earnest, because there was a recent change to Chrome which makes it standard-compliant, but fundamentally different to other browsers and also makes it handle DELETE and other verbs different to how it handles POST and GET.

DELETE redirection

This redirection becomes troublesome if you want to do this in a way which will seamlessly support Ajax calls, as well as standard web requests. It’s troublesome because of a lethal combination of two browser facts.

Firstly, XHR automatically follows redirects. Your JavaScript has no way to jump in and veto or alter any redirect that comes from the server. XHR call fires off to server, server responds with 302 status code and a location header, the browser automatically issues a new request to the specified location.

Secondly, a DELETE request, when it’s redirected, will actually cause another DELETE request to the new location. Yes, if you’re redirecting to the index, congratulations … your server now receives a request to DELETE the fraking index. You lose!!! So the very normal thing to do for a non-Ajax app suddenly becomes an epic tragedy. It’s completely unintuitive, but it’s actually the standard! I learned as much by filing an erroneous Chrome bug. Turns out Chrome’s unintuitive behaviour is actually correct and Firefox and Opera were wrong. Even more confusing, I was seeing POST requests being converted to GETs, and it turns out this is also part of the 302 standard – POSTs can be converted to GETs, but not DELETEs (or PUTs etc I assume).

Just now, I made a little workaround which seems to be working nicely in my case. In my AppController, which is my app’s own subclass of ActionController, I simply convert any 302 response (the default) to a 303, for redirect_to calls. I should maybe do this for XHR calls only, to be “pure” when a normal call comes in, but no real difference anyway.

Finding the new location

While I’m blogging this, let me tell you about another related thing that happens with redirects (this section applies to any type of request – GET, POST, DELETE, whatever). As I said above, XHR will automatically follow requests. The problem is, how do you know where the ultimate response actually came from? XHR will only tell you where the initial request went; it refuses to reveal what it did afterwards. It won’t even tell you if there was any redirect. Knowing the ultimate location is important for HTML5 apps using the History API, because you may want to pushState() to that location. The solution is to have the web service output its own location in a header. Weird that you have to, but gets the job done.

Cross-Origin Resource Sharing makes it possible to do arbitrary calls from a web page to any server, if the server consents. It’s a typical HTML5 play: We could do similar things before, but they were with hacks like JSONP. Cross-Origin Resource Sharing lets us can achieve more and do it cleanly. (The same could be said of Canvas/SVG vs drawing with CSS; WebSocket vs XHR-powered Comet; WebWorker vs yielding with setTimeout; Round corners vs 27 different workarounds; and we could go on.)

This has been available for a couple of years now, but I don’t see people using it. Well, I haven’t checked, but I don’t get the impression many sites are offering their content to external websites, despite social media consultants urging them to be “part of the conversation”. It’s like when people make a gorgeous iPhone app, but their website doesn’t work at all in the same phone (coughfashionhouse) . Likewise, if you’ve got a public API, but not providing JSONP/callback support, it’s not very useful either…making developers host their own cross-domain proxy is tedious. It’s cool there are services like YQL and Embed.ly for some cases, but wouldn’t it be better if web pages could just pull in all that external content directly?

Except in this case, it’s just not happening. Everyone’s offering APIs, but no-ones sharing their content through the web itself. At this point, I should remind you I haven’t actually tested my assumption and maybe everyone is serving their public content with “Access-Control-Allow-Origin: *” … but based on the lack of conversation, I am guessing in the negative. The state of the universe does need further investigation.

Anyway, what’s cool about this is you can treat the web as an API. The Web is my API. “Scraping a web page” may sound dirtier than “consuming a web service”, but it’s the cleaner approach in principle. A website sitting in your browser is a perfectly human-readable depiction of a resource your program can get hold of, so it’s an API that’s self-documenting. The best kind of API. But a whole HTML document is a lot to chew on, so we need to make sure it’s structured nicely, and that’s where microformats come in, gloriously defining lightweight standards for declaring info in your web page. There’s another HTML5 tie-in here, because we now have a similar concept in the standard, microdata.

Now any web page can pull down “http://mahemoff.com/” with a cross-domain XMLHttpRequest. This is fine for a public web page, but something you should be very careful about if the content is (a) not public; or (b) public but dependent on who’s viewing it, because XHR now has a “withCredentials” field that will cause cookies to be passed if it’s on. A malicious third-party script could create XHR, set withCredentials to true, and access your site with the user’s full credentials. Same situation as we’ve always had with JSONP, which should also only be used for public data, but now we can be more nuanced (e.g. you can allow trusted sites to do this kind of thing).

This gives us a cross-domain XHR, for any browser that supports the concept, and it makes a request the usual way, and the request works against my site, but not yours, because of the header I set earlier on my site. Now I can dump that external content in a div:

(This would be a monumentally thick thing to do if you didn’t trust the source, as it could contain script tags with malicious content, or a phishing form. Normally, you’d want to sanitise or parse the content first. In any event, I’m only showing the whole thing here for demo purposes.)

Now comes the fun part: Parsing the content that came back from an external domain. It so happens that I have embedded hCard microformat content at http://mahemoff.com. It’s in the expandable business card you see on the top-left:

It’s based on the hCard microformat, which really just tells you what to call your CSS classes…I told you microformats were lightweight! The whole idea of the card comes from Paul Downey’s genius Hardboiled hCards project.

Anyway, bottom line is we’ve just extracted some content with hCard data in it, so it should be easy to parse it in a standard way and make sense of the content. So I start looking for a hCard Javascript library and find one, that’s the beauty of standards. Even better, it’s called Sumo and it comes from Dan Webb.

The hCard library expects a DOM element containing the hCard(s), so I pluck that from the content I’ve just inserted on the page, and pass that to the library. Then it’s a matter of using the “hCard” object to render a custom UI:

There’s lots more fun to be had with the Web as an API. According to the microformats blog, 2 million web pages now have embedded hCards. Offer that content to the HTML5 mashers of the world and they will make beautiful things.

Share this:

It’s often a requirement for an Ajax app to “bust the cache”, i.e. call a service and ensure its response comes direct and not from a cache. For all the talk of fancy header techniques, the easiest way to do it is by appending an arbitrary parameter, typically a random number and/or a timestamp i.e. call service.com/?random=39583483. I use this pattern not just as an Ajax programmer but also as a user; when I suspect my wiki page or blog is cached, I just add an arbitrary parameter and usually get the most recent result.

With Digg API, that’s impossible. I just tried it and discovered you get an unrecognised argument error if you pass in a parameter they weren’t expecting. This is well-intentioned as it will give early failure advice to callers who mistype or misinterpret parameter names. But unfortunately, it has the major downside of breaking this convention and thus breaking compatibility with a large number of toolkits that exploit this pattern.

I discovered this as I was writing a Google Gadget to show Digg stories and when you fetch content with the Google API _IG_FetchContent("http://digg.com/....", callbackFunc, { refreshInterval: 60 }) – the refresh option apparently leads to Google tacking on an arbitrary parameter called “cache” (it’s impossible to be sure as it’s proxied via Google, but the argument to that proxy is the Digg URL with “cache=” on the end). Net effect: I can’t update Digg stories any more than once per hour. The only way I could do it would be to write my own proxy and cache. A lot more work than the one-liner _IG_FetchContent call! Yeah, like that’s gonna happen.

For the Digg API, perhaps they need to introduce an optional “strict” parameter like a compiler offers to give extra debug info. If it’s off, then let anything through.

A pattern for your consideration, about using Ajax to help pages be RESTful.

Problem

How to personalize content and make pages cacheable and bookmarkable at the same time?

Forces

We want pages to have clean URLs that describe the main content being viewed. Doing so makes pages easily bookmarkable and send-to-friend-able, and also allows us to cache the page anywhere along the way. For example, viewing info about Fight Club should be http://example.com/fightclub and not http://example.com/fightclub/user-mahemoff or http://example.com/fightclub/287490270-1931321-cijE12ZSz

.

We want to personalize pages – say Hi to the user, show them personalized recommendations, etc.

If we personalize, but use the same URL for all users, we break REST and therefore won’t be able to cache any content. My http://example.com/fightclub is different to your http://example.com/fightclub because we each see our own recommendations inside the page.

But if we use diferent URLs for personalization, we can’t cache across users and pages aren’t sent-to-friend-able. If I look up and see http://example.com/fightclub/user-mahemoff, I’m probably not going to bother sending you the URL. Furthermore, my view of the page can’t be cached.

Solution

Create pages generically (same version for all users), and in this generic version, embed a remoting call which will customize the page for the current user. Serve http://example.com/fightclub to everyone. Then everyone’s browser makes a further call to grab custom content (Multi-Stage Download). This additional call is unRESTful as the server will use cookies to decide what content to return, but at least we’ve isolated that component, served the bulk of the content without caching, and given the user something they can bookmark and send to their friends.

Comet Takes to IE Like a Fish Takes to Acid

Comet – or HTTP Streaming, if you will – is a little sensitive when it comes to portability, and I’ll give you four guesses which major browser is causing the grief? Yeah, IE makes it difficult for two reasons: (a) IE’s XMLHttpRequest component doesn’t tell you anything about the response until the connection has closed – even if you try polling it instead of relying on onReadyStateChange, you’ll still get an empty string (Try it); (B) Okay, switch to plan B and inspect IFrame content – we can’t rely on onload, which is only called once at the end, so we must poll. But no, polling won’t help either, as the IFrame content remains empty until (you guessed it) the connection is closed. (Try it).

We’re Going Back, Way Back: Inline Script Tags

Don’t give up on the IFrame so fast … we’re closer than we think. Actually, the solution harkens back to one of the original Push techniques: outputting script tags with inline code (what the HTTP Streaming pattern calls “page streaming”). If you do that in your main document, the code will be executed immediately, and the same is true if you do that inside an IFrame.

So Make the Server Spit Out Inline Script Tags

The portable solution is this: Have the server continuously output script tags that call a known function in the parent frame. When you set the child IFrame’s source to point to this service, it will start evaluating the inline scripts as they pop out of the server. This happens to be one technique people have used for remoting for many years (I think Brent Ashley recently told me he was doing it in ?1999). The twist with Comet is that you keep doing it, and don’t actually close the connection for a while. (Again, I’m sure some people were doing that for a long time too!).

Is it elegant? No. It means the remote service is suddenly coupled to the client-side of the web app – it has to know something about what’s in the parent frame, whereas you’d often like it to be generic and just deliver a response in XML, JSON or whatever. Like most things Ajax, we’re using a hack because it nonetheless push all the right buttons for our users.

See what I mean about coupling? The server-side service had to know that there’s a Javascript function defined in the parent called onNewValue(). At least we’ve minimised the coupling by using an Observer/Event style indirection approach – evidenced by the simple call to “onNewValue()”. It would be worse if it was the script that actually performed application logic directly, repainting the DOM with the new value.

IFrame is the new XMLHttpRequest

Whoever put the X in Ajax ought to put an I in Comit. IE’s XHR component doesn’t provide the response until the connection has closed, and I don’t think version 7 changes that. Assuming (graciously) it happens in IE8, you’ll need to use IFrame for the next five years if you’re doing any Comet work. And of course, you won’t need to do that, because libraries will do it for you.

BTW You could argue that IE is doing the right thing by preventing access to the content until the connection is closed. Maybe they are, maybe they aren’t. From a pragmatic perspective, though, portable Comet requires Ajax. Alternatively, use IFrame with IE and XmlHttpRequest with the others, though I’m not sure if there’s much mileage to be gained from this hybrid strategy.

The feedback has been very helpful and I’ve been able to incorporate it in time for the physical publication – thanks again to everyone in the group.

While listening to the audio, I’ve been taking notes and writing some comments. With the permission of Ralph and Brian, I’m going to be posting these, each discussion as a separate post. It’s an opportunity to see how a group of very intelligent people without much Ajax experience respond to Ajax and the Ajax patterns. You’ll notice two conventions here: “TODO” is a note to myself that some action needs to be taken. “MM” signals my ideas, views, and comments back to the group.

What it's about.
- Probably missing in old browsers if you can't use Ajax on
them.
- Remote call to server without refreshing a whole page.
- I assumed in JS you can open a socket, you could have done
this yourself.
- Depends on what the browser provides. But not
cross-browser. [TODO Mention HTTP restriction and what JS
can do, cf Richer Plugin]
- Ironically, because JS isn't general-purpose (due to
security), it's wound up being a better citizen. Also
because it was kind of low-brow, everyone kind of ignored
it.
- The big idea is this is a way to call the server.

Did you find this pattern easy to understand?
- One thing that troubles me (not particular to this pattern),
pretty soon the code becomes spaghetti-like. Nd good patterns
on how to manage code. [MM Agree, we need a JS patterns
book! That's not the aim here, the book will make that
explicit].
- Part of the problem here is JS itself.
- Haven't seen the word "simple" "elegant" or "pretty" to
describe JS architect. This is a Rube Goldberg solution,
duct-tape ... at it's best. [MM Sort of true, but there's a
lot that can be done to improve it]
- There are libraries that help. (AjaxCaller, Prototype).

Writing here, even though it doesn't address these problems,
could understand/follow it?
- I don't like the Problems section. "How can the browser
communicate with the server?" But this pattern is more
specific than server communication. The next 1-2 patterns
have the same problem.
- But the problem might be the same, but different
Solution. What I dislike is the forces are also the same.
How does it mitigate the forces? Maybe there should be
some different forces.
- Only difference between IFrame and XHR is only restricted
to same host, so maybe okay other than that. [MM
Actually, even this difference doesn't exist, since you
can't read a remote IFrame's content] [TODO Possibly
update forces to be different, back-reflect the solution]

22:30 Doesn't say anything about being asynchronous at the start
- Ajax should be highly responsive. Distributed system, so you
want to minimise communications. ie Must be asynchronous
[TODO Revise forces. HOWEVER, note that XHR doesn't have to
be async.]
- "Conventional means ... slow." He's trying to rule out
Solutions, before we get to the solution. [MM This sort of
goes against the previous suggestion that IFrame and XHR need
different Forces. Maybe suggests different people have
different views on this issue of the forces.] Doesn't so much
talk about the forces as take potshots at existing solutions.
[MM This is a fair point, more prevalent in the initial
patterns as they're arising as a reaction to conventional web
dev, but it's true you don't have to formulate them that
way.]

26:30 It's a long pattern, is that okay?
- Yes, longer than all the others, problem if all patterns were
this long, but given it's so core to Ajax, it's fine.
- Length is fine, but a lot of code there.
- I would like more examples of the old way of doing things.
[Covered in the Anagrams tutorial, perhaps reference it]
- The general idea is if you have a big object and only small
things change at a time, then you keep going back to the
server and grabbing small bits of it. I think of Google
Maps.
- Pattern could be shorter if PHP wasn't written.
- So you'd like less example code, others want more.
- It's a Chimera/Frankenstein ie PHP (or whatever serverside)
on the one side, and even JS is a kind of Frankenstein
language. So it's important to have the PHP, reminds me
that's the game I'm playing. I didn't mind it, seeing all
the pieces together reminds you we're receiving small
chunks etc.
- The pattern is really introducing XHR, not how to use it.
- Disagree with people who are trying to call everything a
pattern.
- Well, these particular patterns are what he calls
foundational. He says they're not really patterns,
they're just how the technology works.
- I know what you're saying about that, and it bugs me too,
but I've been trying to come up with a rationalisation
for admitting that this kind of design exposition is a
contribution to our software architecture literature ...
and if designs recur, if a lot of people come up with the
same solution to a problem ... my mind cries out to keep
distinct from Visitor and ... Composite, but I don't have
the vocab to keep them distinct, and I want to maintain
this notion that the patterns community is talking about
good ideas that keep coming up over and over that we
haven't come up with ourselves ... true, it's a different
kind of discussion from the GoF, without having
disclaimer kind of nags at me [MM Agree wholeheartedly,
why I've sounded a bit apologetic describing these as
patterns, but they just fit into the overall language
well. See earlier blog post.] [TODO Needs better
"disclaimer" in the book]
- So then what are the patterns around XHR?
- Event handling, Asynchronous call
- Lots of people dealing with SOA, problems s.a. async
smell the same but with different names [MM Later
patterns, e.g. Distr'd events]
- Error detection
- Invesion-of-control/DepInj/Observers. People patternising
closures. There's an aspect of dealing with callbacks.
Callbacks are part of the discussion here, and that's an
idea that comes up here.
- Feedback from the call. Or using poll. Fire-and-forget.
Typical remote invocation styles: What does XHR do?
- In JS: Callbacks used here (XHR), also used for user
interops. We want to follow flow-of-control, but if
everything is event handlers, hard to follow. (ie JS hard
to follow because of this style.) A lot of the time,
"callbacks" are basically continuations. It's a general
pattern discussion we could have.

41:30 Real-World Examples and Code.
- These systems are not thoroughly responsive as claimed.
Google Suggest surprisingly fast. Others like Backpackit and
Kiko not. [MM See perf optimisation, also comments in HTML
Message, etc. Alex Kirk mentioned Kiko a while ago as a
problem due to too much browser-side rendering.]
- Need to avoid too many requests
- Curious didn't mention the most famous examples (Google
Suggest etc) Maybe can't see the code. But it's JS, have to
be able to. But maybe unclear. [MM Yes, obfuscated. Also,
tried to avoid the cliche references too much, they're
mentioned elsewhere]
- Discussion about mint stats package, security issues in
uploading data.
- Next, ways of telling your web browser what to report back.
- If trying to keep people from attacking you. It's
interesting to me.
- Applets got crucified for some of the security problems
that JS has. People moved on from panicking about them,
but got away with it because browser wars finished etc.

===============================================================
49:00 IFrame Call
- Poor Man's version of the previous one.
- More like a hack
- What can you do with one that you can't do with the other?
XHR and not with IFr?
- I only know the other way round...IFrame is more compat
with older browsers.
- Long discussion about relative benefits etc.
- Comment about Google using IFrame. Not sure if it's true as
you can zoom in.[TODO More specific about how it's being
used. (I think book version already does that)]
- IFrame Call doesn't talk about hidden frames. [MM XHR
Pattern alternatives has a detailed comparison)] [TODO XHR
comparison should briefly explain how the two are diff,
not just compare the benefits]
- IFrame has no timeout function. (But could fake this using
polling)
- Calls IFrame a hack ... Pot calling the kettle black, since
Ajax itself is sort of a hack. Is it bad just because it's
old? [MM Again, comes down to the comparison. XHR is a
more intention-based, intuitive, API, although it's true
that you won't care about that because you should use a
wrapper lib anyway. Better argument is the extra
functionality such as call tracking.]
- Would like better explanation on what's so bad about
IFrames. [TODO Include x-reference in IFrame Solution.
Also mention the portable wrappers in the Solution, ie you
shouldn't actually care/know which you're using is the more
pertinent issue here.]
- I'm feeling historical. The room we used to sit in is the
room where the original Mosaic web browser developers sat.
Continuously astonished in the web industry, using things
that in ways they weren't invented for. Go down this list,
frames, tables ... Ungodly mess, but really impressed,
poster child for pragmatism and "worse is better". IFrame
is a typical example.
- I didn't get any feel on whether I would use one or the
other.
- Intrigued by the history of JS. Knowing that IFrame predated
XHR because features that made it easier to do the second
came along in 2003. Whether it needs to be here, not sure.
[TODO Sidebar?]

Is it competition for the other two?
- Different problem (push/streaming)
- "How can the browser communicate with the server?" Same
problem as the other two.
- I don't know, it turns the table. Little pattern device
of using the same problem with different solutions may
be okay here. Here, the forces would be different.
Could make the case for having the same problem. [TODO
Change the problem statement, just enough to make it a
little different from previous two.]
What would make you choose this over the others?
- Changes coming from lots of other sources, not just the
client.
- e.g. Chat system.
- I don't want to do this. Don't want to grow the system
...
- But scaleability is oversold. You'll never be like
EBay, don't need to scale up like that. [TODO Good
point, mention in the pattern.]
- Our wiki - 4-5 hits per second - can handle that, but
not scaleable. Wikipedia is not a wiki in that sense,
pages don't change because I believe something like 90%
of all edits are done by 200 people, these are official
wikipedia authors. Specific process.
- Doesn't fit into proxies. Caching etc doesn't ddeal
with longer responses.
Long refactoring illustration here. Do the others have one?
- Streaming wiki demo.
- Seemed nice to refactor in this way.
- The group is looking at the live version on the web -
"There's about a half dozen laptops in it for the
author's information". It worked. [MM Phew]

===============================================================
1:20
- I didn't get the feel of which one to use.
- Was hidden in the Alternatives section [MM TODO Emphasise
this in the partintro, maybe in the solutions]

======
Next time: Different patterns
- Problem with web version [MM Incidentally, the wiki
publishing hasn't worked out ideally, didn't get the full
benefit as I didn't open up the wiki (and still haven't due
to spam). Ideally would have better PS format. (Blogged
about printing from the web a yr ago as it happens.)]
- Being on the web doesn't seem to affect (ie drop) sales.
[MM We'll soon find out...:-)]

The feedback has been very helpful and I’ve been able to incorporate it in time for the physical publication – thanks again to everyone in the group.

While listening to the audio, I’ve been taking notes and writing some comments. With the permission of Ralph and Brian, I’m going to be posting these, each discussion as a separate post. It’s an opportunity to see how a group of very intelligent people without much Ajax experience respond to Ajax and the Ajax patterns. You’ll notice two conventions here: “TODO” is a note to myself that some action needs to be taken. “MM” signals my ideas, views, and comments back to the group.

What it's about.
- Probably missing in old browsers if you can't use Ajax on
them.
- Remote call to server without refreshing a whole page.
- I assumed in JS you can open a socket, you could have done
this yourself.
- Depends on what the browser provides. But not
cross-browser. [TODO Mention HTTP restriction and what JS
can do, cf Richer Plugin]
- Ironically, because JS isn't general-purpose (due to
security), it's wound up being a better citizen. Also
because it was kind of low-brow, everyone kind of ignored
it.
- The big idea is this is a way to call the server.

Did you find this pattern easy to understand?
- One thing that troubles me (not particular to this pattern),
pretty soon the code becomes spaghetti-like. Nd good patterns
on how to manage code. [MM Agree, we need a JS patterns
book! That's not the aim here, the book will make that
explicit].
- Part of the problem here is JS itself.
- Haven't seen the word "simple" "elegant" or "pretty" to
describe JS architect. This is a Rube Goldberg solution,
duct-tape ... at it's best. [MM Sort of true, but there's a
lot that can be done to improve it]
- There are libraries that help. (AjaxCaller, Prototype).

Writing here, even though it doesn't address these problems,
could understand/follow it?
- I don't like the Problems section. "How can the browser
communicate with the server?" But this pattern is more
specific than server communication. The next 1-2 patterns
have the same problem.
- But the problem might be the same, but different
Solution. What I dislike is the forces are also the same.
How does it mitigate the forces? Maybe there should be
some different forces.
- Only difference between IFrame and XHR is only restricted
to same host, so maybe okay other than that. [MM
Actually, even this difference doesn't exist, since you
can't read a remote IFrame's content] [TODO Possibly
update forces to be different, back-reflect the solution]

22:30 Doesn't say anything about being asynchronous at the start
- Ajax should be highly responsive. Distributed system, so you
want to minimise communications. ie Must be asynchronous
[TODO Revise forces. HOWEVER, note that XHR doesn't have to
be async.]
- "Conventional means ... slow." He's trying to rule out
Solutions, before we get to the solution. [MM This sort of
goes against the previous suggestion that IFrame and XHR need
different Forces. Maybe suggests different people have
different views on this issue of the forces.] Doesn't so much
talk about the forces as take potshots at existing solutions.
[MM This is a fair point, more prevalent in the initial
patterns as they're arising as a reaction to conventional web
dev, but it's true you don't have to formulate them that
way.]

26:30 It's a long pattern, is that okay?
- Yes, longer than all the others, problem if all patterns were
this long, but given it's so core to Ajax, it's fine.
- Length is fine, but a lot of code there.
- I would like more examples of the old way of doing things.
[Covered in the Anagrams tutorial, perhaps reference it]
- The general idea is if you have a big object and only small
things change at a time, then you keep going back to the
server and grabbing small bits of it. I think of Google
Maps.
- Pattern could be shorter if PHP wasn't written.
- So you'd like less example code, others want more.
- It's a Chimera/Frankenstein ie PHP (or whatever serverside)
on the one side, and even JS is a kind of Frankenstein
language. So it's important to have the PHP, reminds me
that's the game I'm playing. I didn't mind it, seeing all
the pieces together reminds you we're receiving small
chunks etc.
- The pattern is really introducing XHR, not how to use it.
- Disagree with people who are trying to call everything a
pattern.
- Well, these particular patterns are what he calls
foundational. He says they're not really patterns,
they're just how the technology works.
- I know what you're saying about that, and it bugs me too,
but I've been trying to come up with a rationalisation
for admitting that this kind of design exposition is a
contribution to our software architecture literature ...
and if designs recur, if a lot of people come up with the
same solution to a problem ... my mind cries out to keep
distinct from Visitor and ... Composite, but I don't have
the vocab to keep them distinct, and I want to maintain
this notion that the patterns community is talking about
good ideas that keep coming up over and over that we
haven't come up with ourselves ... true, it's a different
kind of discussion from the GoF, without having
disclaimer kind of nags at me [MM Agree wholeheartedly,
why I've sounded a bit apologetic describing these as
patterns, but they just fit into the overall language
well. See earlier blog post.] [TODO Needs better
"disclaimer" in the book]
- So then what are the patterns around XHR?
- Event handling, Asynchronous call
- Lots of people dealing with SOA, problems s.a. async
smell the same but with different names [MM Later
patterns, e.g. Distr'd events]
- Error detection
- Invesion-of-control/DepInj/Observers. People patternising
closures. There's an aspect of dealing with callbacks.
Callbacks are part of the discussion here, and that's an
idea that comes up here.
- Feedback from the call. Or using poll. Fire-and-forget.
Typical remote invocation styles: What does XHR do?
- In JS: Callbacks used here (XHR), also used for user
interops. We want to follow flow-of-control, but if
everything is event handlers, hard to follow. (ie JS hard
to follow because of this style.) A lot of the time,
"callbacks" are basically continuations. It's a general
pattern discussion we could have.

41:30 Real-World Examples and Code.
- These systems are not thoroughly responsive as claimed.
Google Suggest surprisingly fast. Others like Backpackit and
Kiko not. [MM See perf optimisation, also comments in HTML
Message, etc. Alex Kirk mentioned Kiko a while ago as a
problem due to too much browser-side rendering.]
- Need to avoid too many requests
- Curious didn't mention the most famous examples (Google
Suggest etc) Maybe can't see the code. But it's JS, have to
be able to. But maybe unclear. [MM Yes, obfuscated. Also,
tried to avoid the cliche references too much, they're
mentioned elsewhere]
- Discussion about mint stats package, security issues in
uploading data.
- Next, ways of telling your web browser what to report back.
- If trying to keep people from attacking you. It's
interesting to me.
- Applets got crucified for some of the security problems
that JS has. People moved on from panicking about them,
but got away with it because browser wars finished etc.

===============================================================
49:00 IFrame Call
- Poor Man's version of the previous one.
- More like a hack
- What can you do with one that you can't do with the other?
XHR and not with IFr?
- I only know the other way round...IFrame is more compat
with older browsers.
- Long discussion about relative benefits etc.
- Comment about Google using IFrame. Not sure if it's true as
you can zoom in.[TODO More specific about how it's being
used. (I think book version already does that)]
- IFrame Call doesn't talk about hidden frames. [MM XHR
Pattern alternatives has a detailed comparison)] [TODO XHR
comparison should briefly explain how the two are diff,
not just compare the benefits]
- IFrame has no timeout function. (But could fake this using
polling)
- Calls IFrame a hack ... Pot calling the kettle black, since
Ajax itself is sort of a hack. Is it bad just because it's
old? [MM Again, comes down to the comparison. XHR is a
more intention-based, intuitive, API, although it's true
that you won't care about that because you should use a
wrapper lib anyway. Better argument is the extra
functionality such as call tracking.]
- Would like better explanation on what's so bad about
IFrames. [TODO Include x-reference in IFrame Solution.
Also mention the portable wrappers in the Solution, ie you
shouldn't actually care/know which you're using is the more
pertinent issue here.]
- I'm feeling historical. The room we used to sit in is the
room where the original Mosaic web browser developers sat.
Continuously astonished in the web industry, using things
that in ways they weren't invented for. Go down this list,
frames, tables ... Ungodly mess, but really impressed,
poster child for pragmatism and "worse is better". IFrame
is a typical example.
- I didn't get any feel on whether I would use one or the
other.
- Intrigued by the history of JS. Knowing that IFrame predated
XHR because features that made it easier to do the second
came along in 2003. Whether it needs to be here, not sure.
[TODO Sidebar?]

Is it competition for the other two?
- Different problem (push/streaming)
- "How can the browser communicate with the server?" Same
problem as the other two.
- I don't know, it turns the table. Little pattern device
of using the same problem with different solutions may
be okay here. Here, the forces would be different.
Could make the case for having the same problem. [TODO
Change the problem statement, just enough to make it a
little different from previous two.]
What would make you choose this over the others?
- Changes coming from lots of other sources, not just the
client.
- e.g. Chat system.
- I don't want to do this. Don't want to grow the system
...
- But scaleability is oversold. You'll never be like
EBay, don't need to scale up like that. [TODO Good
point, mention in the pattern.]
- Our wiki - 4-5 hits per second - can handle that, but
not scaleable. Wikipedia is not a wiki in that sense,
pages don't change because I believe something like 90%
of all edits are done by 200 people, these are official
wikipedia authors. Specific process.
- Doesn't fit into proxies. Caching etc doesn't ddeal
with longer responses.
Long refactoring illustration here. Do the others have one?
- Streaming wiki demo.
- Seemed nice to refactor in this way.
- The group is looking at the live version on the web -
"There's about a half dozen laptops in it for the
author's information". It worked. [MM Phew]

===============================================================
1:20
- I didn't get the feel of which one to use.
- Was hidden in the Alternatives section [MM TODO Emphasise
this in the partintro, maybe in the solutions]

======
Next time: Different patterns
- Problem with web version [MM Incidentally, the wiki
publishing hasn't worked out ideally, didn't get the full
benefit as I didn't open up the wiki (and still haven't due
to spam). Ideally would have better PS format. (Blogged
about printing from the web a yr ago as it happens.)]
- Being on the web doesn't seem to affect (ie drop) sales.
[MM We'll soon find out...:-)]

Abe Fettig’s done some important experimenting to arrive at a direct remoting technique, one which bypasses the need for a Cross-Domain Proxy and doesn’t rely on cross-domain On-Demand Javascript. Compared to the latter technique, Abe’s idea is more functional, because you get the power, expressivity, and bidirectional capability of XMLHttpRequest, as opposed to the On-Demand Javascript hack, which only allows downloading, though you could perhaps pass CGI arguments with the script request, or use one of the image/stylesheet hacks to get information in the other direction.

If we can bypass the server. then we can consider the idea of Host-Proof Authentication. It’s based on Richard Schwartz’s Host-Proof Hosting idea, where encrypted data is decrypted on the fly in the browser. In similar vein, if you needed third-party authentication, these remoting hacks are one way to keep your password away from the prying eyes of the server host. A while back, one of the internet banks (egg?) copped it for asking users to give them all their cusomter IDs, passwords, etc., so they could provide a one-stop-shop service. Maybe Host-Proof Authentication would be a better approach – if not automated, a portal could be set up to allow users to shuffle funds around within the browser.

Back here on Earth, I wouldn’t in reality use Host-Proof Authentication for a critical application – not without a lot more consideration – because there are two reality checks:

Host-Proof Hosting is far from perfect– Alex Russell has noted it’s vulnerable to script injection attacks. See the comments in the above links for more on that. Similar comments apply to Host-Proof Authentication.

All these direct remoting techniques rely on some form of co-operation with the external server. e.g. Abe’s technique requires it to explicitly set the document.domain property; On-Demand Javascript requires it to expose appropriate scripts; image remoting requires the script to recognise any variables and output an invisible pixel; etc. The external API would have to explicitly let the browser perform remote authentication.

If Ajax apps are to be rich, there must be a way for the server to pass new information to the browser. For example, new stock quotes or an instant message someone else just sent you. But the browser’s not a server, so the server can’t initiate an HTTP connection to alert the browser. The standard way to deal with this dilemma is Periodic Refresh, i.e. having the browser poll the server every few seconds. But that’s not the only solution.

The recent podcast on Web Remoting includes a discussion of the HTTP Streaming pattern. By continuing to stream information from the server, without closing the connection, you can keep the browser content fresh. I wasn’t aware that it was being used much on the public web, since it can be costly, but I recently discovered JotLive (which is only semi-public since it requires registration) is indeed using it. Do you know any other examples?

We’re using a (very slightly modified) version of LivePage, which Donovan Preston wrote as part of Nevow, a Python library for building web applications using the Twisted networking framework (which I just wrote a book on: Twisted Network Programming Essentials). LivePage doesn’t use polling. Instead, it uses a clever technique where each browser keeps an open XMLHTTP request to the server at all times, opening a new connection each time the old one closes. That way every client viewing the page is constantly waiting for a response from the server. When the server wants to send a message to a client, it uses the currently open request. So there’s no waiting.

Alternative Pattern:Periodic Refresh is an obvious alternative to HTTP Streaming. It fakes a long-lived connection by frequently polling the server. Generally, Periodic Refresh is more scaleable and easier to implement in a portable, robust, manner. However, HTTP Streaming can deliver more timely data, so consider it for systems, such as intranets, where there are less simultaneous users, you have some control over the infrastructure, and each connection carries a relatively high value.

Refactoring Illustration: The Basic Wiki Demo, which uses Periodic Refresh, has been refactored to use [http://ajaxify.com/run/wiki/streaming](HTTP Streaming).

Solution:Stream server data in the response of a long-lived HTTP connection. Most web services do some processing, send back a response, and immediately exit. But in this pattern, they keep the connection open by running a long loop. The server script uses event registration or some other technique to detect any state changes. As soon as a state change occurs, it pushes new data to the outgoing stream and flushes it, but doesn’t actually close it. Meanwhile, the browser must ensure the user-interface reflects the new data. This pattern discusses a couple of techniques for Streaming HTTP, which I refer to as “Page Streaming” and “Service Streaming”.
“Page Streaming” involves streaming the original page response. Here, the server immediately outputs an initial page and flushes the stream, but keeps it open. It then proceeds to alter it over time by outputting embedded scripts that manipulate the DOM. The browser’s still officially writing the initial page out, so when it encounters a complete <script> tag, it will execute the script immediately. A simple demo is available at http://ajaxify.com/run/streaming/.
…(illustration and problems)…
“Service Streaming” is a step towards solving these problems, though it doesn’t work on all browsers. The technique relies on XMLHttpRequest Call (or a similar remoting technology like IFrame_Call). This time, it’s an XMLHttpRequest connection that’s long-lived, instead of the initial page load. There’s more flexibility regarding length and frequency of connections. You could load the page normally, then start streaming for thirty seconds when the user clicks a button. Or you could start streaming once the page is loaded, and keep resetting the connection every thirty seconds. Having a range of options helps immeasurably, given that HTTP Streaming is constrained by the capabilities of the server, the browsers, and the network.
…
Experiments suggest that the Page Streaming technique does work on both IE and Firefox ([1]), but Service Streaming only works on Firefox, whether XMLHTTPRequest ([2]) or IFrame ([3]) is used. In both cases, IE suppresses the response until its complete. You could claim that’s either a bug or a feature; but either way, it works against HTTP Streaming.
…
Donovan Preston explains the technique he uses in Nevow, which overcomes this problem:

When the main page loads, an XHR (XMLHttpRequest) makes an “output conduit” request. If the server has collected any events between the main page rendering and the output conduit request rendering, it sends them immediately. If it has not, it waits until an event arrives and sends it over the output conduit. Any event from the server to the client causes the server to close the output conduit request. Any time the server closes the output conduit request, the client immediately reopens a new one. If the server hasn’t received an event for the client in 30 seconds, it sends a noop (the javascript “null”) and closes the request.

Share this:

G’Day

Welcome to Michael Mahemoff's blog, soapboxing on software and the web since 2004. I'm presently using HTML5 and the web to make podcasts easier to share, play, and discover at Player FM. I've previously worked at Google and Osmosoft, and built the Ajax Patterns wiki and corresponding book, "Ajax Design Patterns" (O'Reilly 2006).
For avoidance of doubt, I'm not a female, nor ever have been to my knowledge. The title of this blog alludes to English As She Is Spoke, a book so profoundly flawed it reminded me of the maturity of the software industry when this blog began in 2004. I believe the industry has become more sophisticated since then, particularly the importance of UX.
Follow @mahemoff