In that last post, I walked through the PAuthorLib.dll and showed you how to crawl
through your event log error messages and create a pivot collection. The result
of that initial effort was a nice view into our events:

While this certainly got the job done and is a very powerful and
compelling view into our data, we need to realize that as our data grow, the amount
of entries in our linked collection is limited. From
the Developer Overview, we see that the maximum number of items we should have
in a single collection is 3,000:

So, while a simple collection will get the job done for your smaller amounts of data,
you will really run into some challenges with your larger datasets like our ASP.NET
event log error messages. To combat this limitation you can create what’s called
a Linked Collection. The idea is that it’s just a way for you to link together
related collections in order to provide a seamless experience for your users.
In our case, a natural break for our collections will be based upon the exception
type with a summary collection and then a separate collection for each exception type.
If I were to draw this out:

Event Log Header Collection Source

The idea behind this structure is that the Exception summary would simply link to
each of these exception collections. First, we’ll create a colleciton source
for our exception summary. As
in our previous collection source (in my last blog post), we inherit from the
AbstractCollectionSource class and use the LoadHeaderData() method to add our facet
categories to the collection. In this case, we’ll create two categories – the
number of Occurrences and the Urls where the exception occurred. Another difference
is that we are going to pass the already parsed collection of messages into the constructor.
The reason for that is so we don’t have to repeat the parsing of the event log messages
multiple times.

Then, in the LoadItems() method, we provide the logic to generate the PivotItem collection.
The one key item to make note of is the use of the Href property
of the PivotItem object. This is where we specify the collection we wish to
link to this item. Since each of the PivotItems will be a summary of the number
of each exception type – we’ll name the sub-collections by its exception type.
For example, NullReferenceException.cxml, SqlException.cxml, etc.

Event Log Collection Source Redux

Previously, when we generated the pivot collections, we were outputting all of the
records into a single collection. Now that we are generating a collection for
each exception type, we will need to put a filter in our exception collection and
then incorporate that filter into our item generation. Other than that, the
code we wrote previously remains largely unchanged, so I left the majority of it out
and only included the snippets that we care about below.

Generate and test the collection

Then, the only thing we have left to do is generate and test our linked collections.
I won’t go into a lengthy explanation of how we generate the collections because I
did that in the last blog entry. I will show the broad strokes required
to tie this all together, though:

Once we we have this code and everything has been generated, if we open the output
folder, we’ll see the following structure:

We see our ExceptionSummary pivot collection and all of the deep zoom folders.
So, when we open the Pivot tool, we’ll see a nice parent collection:

This gives us a nice breakdown of the number of occurrences for each exception in
our source data. Immediately we see an outlier (more on that later) between
the 6,000 and 7,000 item mark and when we select that tile, we see the following:

We also see a green “Open” box (surrounded in a red rectangle, my emphasis) which
links to our NullReferenceException.cxml. When we click that box, the tool will
immediately open that collection in the UI for our perusal – providing a very similar
look to what we saw in the last entry:

Closing Thoughts

Now, you may have noticed a contradiction above. I said that a collection should
have no more than 3,000 items and yet, with the NullReferenceException collection,
we saw in the summary that it had over 6,000 items. That is a very good point
and will be a subject of a future blog post. I wanted to illustrate the simple
collections and the linked collections before we got into that third type of collection
from above – the Dynamic Collection. Stay tuned!