Building the Realtime User Experience 102
rheotaxis writes "Many professional web developers have spent years building dynamic, database-driven web applications, but some of us, like myself, want to make the user experience more interactive and instantaneous. The book Building the Realtime User Experience, by Ted Roden, is an introduction to some new techniques making that happen now. New web servers like Cometd and Tornado power solutions that keep HTTP connections open until data is available for the clients requesting it, a technique called 'long-polling.' This means web developers can provide a real-time user experience using HTTP for all sorts of client devices now connecting to the Internet, not just web browsers, but mobile devices as well." Read below for the rest of rheotaxis's review.
This book covers SUP and PubSubHubbub syndication, messaging with Bayeux protocol and Cometd, and asynchronous Python using Tornado, contrasting these with well-known client-side JavaScript methods. It then demonstrates how long-polling can implement and integrate chat, IM, SMS, and analytics. The last chapter wraps up with an example using all these technologies, a multi-user, real-time, interactive game using geo-location with mobile clients. Ted's writing style is concise and to the point, focused exclusively on the challenges presented and solved in each chapter, including just enough details for experienced programmers to download and setup the software tools being used, including the Google App Engine. The code samples are straight-forward, but be forewarned, it will be easier for readers with some experience building server-side scripts like PHP, Python, or Java, and a database server like MySQL. On the other hand, even if you never used Google App Engine before, that's OK, because Ted covers that in enough detail to get you started quickly. The sample code wasn't yet available on the O'Reilly web site, so you'll need to type in the code samples to try them. Check the O'Reilly errata page for the book to get a head start making the code work. (Full disclosure: I posted some of the errata.) The sample code for Cometd and Tornado ran easily on my laptop (HP 2.2 GHz with Windows Vista), and should be fine on Linux or Mac. Everything you need is open-source and easily downloaded.
| Building the Realtime User Experience | |
| author | Ted Roden |
| pages | 320 |
| publisher | O'Reilly Media |
| rating | 8/10 |
| reviewer | rheotaxis |
| ISBN | 0596806159 |
| summary | Shows you how to build realtime user experiences by adding features on your site without making big changes to the existing infrastructure |
The author explains that real-time web development puts the user at the center of all web interactions, and that developers have struggled with solving the push versus pull problem. The pull method requires multiple, periodic queries for updates from server information feeds, something that wastes server CPU and bandwidth when no changes have occurred, and is compounded by the number of different users making these queries. The push method allows the servers to contact the clients when information feeds have been updated, saving CPU and bandwidth.
RSS was designed for easy syndication of information feeds, but it suffers from the limitations of the pull methodology. While several push technologies have been proposed to solve this problem, only Simple Update Protocol (SUP) and PubSubHubbub are covered in detail here. Both of these are demonstrated with PHP code, so they should be easy to implement on hosted web account with PHP and MySQL. The author explains that while SUP isn't a real push methodology, it does address some the CPU and bandwidth issues. PubSubHubbub, a true push methodology when compared to SUP, is described with an equal amount of detail.
Next, the book covers techniques already familiar to JavaScript programmers who have experience building AJAX enabled web pages. Skim the text and glance at the code and diagrams in Chapter 3, if you already have this experience. The subtitle for this chapter is "Widgets in Pseudorealtime", and the key take away from this chapter is that client-side JavaScript can be used with pull or push technologies, depending upon the server-side implementation. If you don't yet have experience with AJAX, then be sure you can follow these code examples, because AJAX will be used in all the other chapters.
Have you ever wished your blog could send live updates to your readers the moment you post them? You'll learn how, using Bayeux protocol, Java, Cometd, and the Jetty web server. The sample code allows you to grasp how long-polling works with modern browsers. Once a client browser opens an HTTP connection to a web server using a POST method, the server leaves this connection open until it has data to deliver to the client. This chapter suggests using Firebug, a Firefox plugin for debugging web applications from the client side, to discover and track long-polling seasons.
Do you need to handle a large amount of incoming data, and then redisplay it on client browsers with almost no delay? Tornado, the Python web server, provides a solution. Tornado was created by FriendFeed, and made open source after being acquired by Facebook. Kudos to Facebook for making Tornado available. Please read Chapter 5 and 6 together, since they both explain how the Tornado server works. The sample code starts with Python threads that cache a Twitter feed, process and filter it, then send it out to web browsers already connected to Tornado using long-polling and asynchronous callbacks. Tornado is then used to implement a peer to peer chat system using long-polling. Again, each client stays connected to the Tornado server until messages are ready to deliver to each chat participant. Taken together, Chapter 5 and 6 lay the groundwork for more advanced Tornado web applications covered later in the book.
This is followed by two chapters using the Google App Engine to support real-time user experiences even though the Google App Engine does not support long-polling. If you have never used Google App Engine before this, don't worry. The author spends 10 pages explaining how to sign up. Then you build an application in the cloud and connect with your IM client, instead of the web browser. You can make your IM server accept commands and respond with information from other web services. The section, "Setting Up an API", gives you a tantalizing glimpse of possibilities explored later in the book. After adding Python code from the next chapter, you have SMS capabilities. Why would you want to do this? Because it allows users to keep informed while they're away from the web, making SMS another part of the real-time user experience.
Once you have implemented and deployed your real-time application, you can add analytics that give you immediate feedback about user interactions with your site. Instead of paying for a service, you can build your own custom web analytics using Tornado and client-side JavaScript. I especially like the authors approach to summarizing all the incoming web usage data into a single, super-simple, HTML template that is immediately updated as web usage changes. It should satisfy your curiosity to watch users interact with your web site in real-time, and you can make it track IM and SMS traffic connecting to your server too. Finally, the last chapter demonstrates how all the know-how you learn from the rest of the book can be combined in new and highly imaginative ways. The author provides all the details you need to setup a location-based, multi-user, real-time, interactive, game played by users with mobile web devices.
This book would be good for anyone that needs to quickly learn how to use Tornado and integrate it with other web services. It's also helpful for people who want to integrate the Google App Engine with other web services. Whether you're going to build a real-time web experience from the ground up, or just add a few more dynamic features to an existing site, the lessons you can learn from this book will help you.
You can purchase Building the Realtime User Experience from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Re:I didn't read the whole thing (Score:5, Informative)
Why don't they use straight sockets and TCP/IP? Why does it have to be HTTP?
Straight sockets and TCP/IP in client-side Javascript?
The point is to make it work with Javascript/AJAX. Since you can’t open an arbitrary port and listen, you open an XMLHttpRequest and wait for data.
Server push and web sockets... (Score:0, Informative)
Server side push never took off, web sockets [w3.org] probably will.
That's all anyone need know.
Re:cost of keeping a socket open? (Score:5, Informative)
Hi, Ted Roden here, author of this book.
I definitely spend a lot of time talking about that issue in the book.The reviewer (understandably) didn't write about all of the pros/cons and content in the book.
I talk about how many servers (apache) are really bad at keeping these types of connections open and suggest using servers that were specifically designed for this (tornado). I also spend a while talking about how to get them all working together so you don't have to rearchitect everything just to get something set up.
-Ted
Re:But apache isn't at fault. (Score:4, Informative)
All good points, and I agree. In the book, I spend a lot of time talking about technologies that aren't HTTP. I was just responding to the points about HTTP (above).
Re:I didn't read the whole thing (Score:3, Informative)
Well that's annoying, because when I posted (http://slashdot.org/comments.pl?sid=1810648&cid=33809274) that it's not surprising that W3C says not to use HTML5, I got a +5 response to my comment saying we should use it anyways. So which is it?
The three people who modded up that comment first ran a poll of the entire Slashdot population, verified that 95% of users agree with it, and only then proceeded to mod up that comment. This way we can be sure that those three moderators and the opinions expressed in that comment fully represent all Slashdot users (except the remaining 5%, who are pussies).