After I posted my last question here, I read a bunch of articles about how treeviews are the devil and stuff. Almost everyone talks about better search engines, tagging and faceted search.

But it happens, that I need an explorative Interace for a big hierarchical data structure.

Here are the key points:

The data gets generated hierarchically, you can request child nodes, but you never know how many children a node has, can be zero, can be thousands.

When the users use the system for the first time, they tell you that they need ALL the data, because it's their business and telling them, that "smaller pieces" are better to comprehend is not an option.

When they use the system for a time and understand that they only need some of the data they tend to filter it down to the stuff they need, but this is a learning process and most of them only realize what they need when they tried it.

Because of the exploration phase, which happens the first few usage weeks/months a search-like feature isn't possibile. Also, because the data is big, generating tags out of it isn't possible. Even a faceted search property can have thousands of characteristics.

Building a simple tree would allow to browse the whole data, but the amount of child-nodes a node can have can be enough to kill the client.

I thought about a two phase interface:

Exploration View

Specific View

The last one would consist of the the nodes a user found and deemed interesting.

But the first one needs some mechanisms, which allow to browse the whole data in comprehensible data chunks.

My ideas were

a paging for every node, if it has children. so the user only sees 10 children and can browse to the next 10 or to a specific page. Probably with the help of sorting and/or filtering.

a virtual node, which loads the next 10 nodes, but this would only slow the user down AND let the tree get big anyway, so it's probably a bad idea.