A Peek Under the Hood of YUI's DataTable Control in 2.5.0

We think the 2.5.0 release of the YUI DataTable Control is our best one yet, so we wanted to take a moment to look back at some of the challenges we've faced over the past few months and to peek under the hood at some of the techniques we've used to tackle these problems.

A Yielding Render Queue

The browser environment presents non-trivial challenges for application developers, including security sandboxes and limits to CPU and memory. The DataTable is a large and robust application even before you load it up with data, and one of the core challenges for us has been to improve performance as more and more data is brought into the table. In previous releases, the browser's UI thread would tend to lock up after a certain data-size threshold was crossed as DataTable churned through the management of its internal objects and the DOM.

We've made huge improvements in DataSource to help speed things up on the data processing side, but there remained the issue of rendering all this data to the UI. Modifying the DOM can get expensive quickly, especially when you're talking about drawing or updating hundreds of rows with several cells per row.

Browsers are impatient; they like to get to work updating the display as soon as something is changed in the DOM. While we appreciate the enthusiasm, this model can be counterproductive when a lot of changes need to be made — such as rendering a DataTable with hundreds of rows. After each DOM change, the browser prepares to update the UI, but it won't actually redraw the UI until the JavaScript has finished executing. In effect, all the preparatory work done between subsequent DOM updates needlessly pulls resources away from the remaining work that needs to be done. All the while, the UI sits there unchanged.

To address this issue, we've introduced progressive rendering to the new version of DataTable.

By default, progressive rendering isn't turned on, as the benefit is mostly evident for tables in excess of 50 rows. To enable progressive rendering, set the DataTable's renderLoopSize configuration to the number of rows you want rendered per iteration. For example, to configure your DataTable to draw 20 rows at a time:

Supposing myHugeDataSet contains 412 records, the DataTable will immediately render itself in an empty state, then milliseconds later 20 rows will appear, then 20 more rows, and so on until all of the 412 rows are displayed.

Scrolling This Way and That Way

The scrollability feature, which was too fragile in previous releases, warranted a markup overhaul to give DataTable a more solid foundation to support fixed-header scrolling plus horizontal scrolling. Our challenge was to achieve a stable xy-scrolling mechanism while remaining accessible to screenreaders. (Screenreader software, which assists blind or partially sighted users, is good at handling <table> elements and their contents; it doesn't do well with tabular data that's marked up in other ways [such as in <div>s]. For a general introduction to screenreaders, check out this YUI Theater video.)

To start, the simplest way to achieve scrolling was to split DataTable markup into two <table> elements each housed in their own containers with their own overflow styles. Next, because of our built-in support for nested headers, we wanted to keep <td> "header" attributes as the best technique to allow screenreaders to make sense of our content. Trouble was, without <th> elements, these "header" attributes were meaningless. We were forced to duplicate all the markup of the <thead> of the first <table> in another <thead> in the second <table>. A quick absolute-positioning of the second <thead> to the far-left places these meant-for-screenreaders-only elements offscreen.

A Fresh Look at Pagination

Pagination is such a fast way to reduce the footprint of a large dataset that we found ourselves recommending this feature to a lot of our implementers. Problem was, everybody seemed to have different expectations when it comes to pagination. The challenge here was to create a solution simple enough for the plug-and-play needs of some, but flexible enough for implementers with highly customized requirements. The result of our efforts is the new YAHOO.widget.Paginator class (currently packaged with the DataTable build).

Before we tell the interesting story, here’s the less interesting one. To paginate your DataTable, all your Paginator needs to know is how many rowsPerPage you want to display. The rest it can figure out on its own.

Of course, dividing a total number of records by a set number of rows per page is simple math. That part is easy to code. It’s in the UI implementation that things get dicey.

To tackle this problem, Paginator is build on a template and UI component system. The template config contains the markup describing how the pagination controls should be rendered. The default template looks like this:

Take a look at the classes under the YAHOO.widget.Paginator.ui namespace in DataTable’s API docs for a list of each component’s options.

Of course, if none of the pre-packaged UI components suits your needs, you can create your own components. Just drop them in the YAHOO.widget.Paginator.ui namespace and you can immediately reference them by name in your Paginator template and config. And if you create a cool Paginator component, let us know in the comments or on the YUI community forum — we'd love to see what you build with this, as would other DataTable users.

Columns Get Their Day

Version 2.5.0 introduces a new set of APIs for managing Columns in the DataTable: setting widths, hiding and showing, inserting, removing, and drag-and-drop reordering of Columns are now supported out of the box. Hiding and showing of Columns is implemented by setting a column's width to "1px" on hide and and reverting to the original width on show. Inserting a Column requires passing in an object literal Column definition. This will create a new Column instance, add it to the internal ColumnSet, and update the DOM as necessary. Removing a Column will remove the Column instance from the ColumnSet as well as removing all related elements from the DOM. Drag-and-drop reordering of Columns can be enabled for the entire DataTable via the constructor config, as long as the Drag and Drop Utility is available on the page. When a Column is reordered, it is first removed from the DataTable and then inserted into the new position.

Since Columns can be assigned a width at instantiation or dynamically at runtime, users can easily resize their Column widths and then save their settings as a preference for their next visit using the Cookie Utility that Nicholas C. Zakas contributed to YUI for the 2.5.0 release. Here's one way you might achieve that:

There is No "Rest" in Iteration

2.5.0 is a big release for DataTable, but our work is not done, and we're already tackling our next set of top issues, including better support of server-side sort and pagination, more performance enhancements with dynamically added and removed data, and more robust screenreader accessibility for dynamic states.

We want to take this chance to thank all the members of the community who have taken the time to implement our products, file bugs, suggest features, and contribute to the forum (this means you, Satyam!). Your participation is wholly appreciated and makes an incredible difference in what we do and how we do it.