Note that although these are aggregates, the measure (eg, Error Rate) is derived
at the step level. The aggregate is then just the mean of all values.

To change the measure (top axis):

Hover your cursor over the top axis label (the range selection).
A pop-up menu will appear.

Select a new measure from the list.

To change the aggregation type (side axis):

Hover your cursor over the left-side axis label (the domain selection).
A pop-up menu will appear.

Select a new aggregation type from the list.

To see more detail on bars in the graph, hover over them with
your mouse.

As with other reports in DataShop, your current view is based on
the selected sample, knowledge components, models, students, and problems.
Toggling any of these will cause the current view to update.

Comparing across samples

If you select more than one sample, the Performance Profiler
will render a graph for each sample.

DataShop orders rows in the graphs based on the Sort By type and the Order (ascending or descending), selected
in the left-hand navigation bar. Sorting occurs in each
graph independently, so to compare the same items across samples,
select them in the navigation sidebar and/or create a knowledge component set.

Setting graph limits

You may want to set upper or lower limits on the quality of items
being aggregated. For example, you may want to see only the top 10
error rates for students in a large dataset; or the bottom five "average
number of incorrects" for steps. In either case, the terms "top"
and "bottom" reflect an assessment of the measure (eg, top error rate,
and bottom average number of incorrects), and mean slightly different
things depending on the currently selected measure and sort order.
The table below attempts to clarify these interpretations:

“Top” and “Bottom” defined by measure
and an “ascending” order

Measure

Top

Bottom

Error rate (%) /Predicted error rate

Lowest error rate (%)

Highest error rate (%)

Residuals

Items with no residual, followed by most under-predicted error rate (predicted below actual)

Most over-predicted error rate (predicted above actual)

Assistance Score

Lowest assistance score

Highest assistance score

Total Hints

Fewest hint requests

Most hint requests

Total Incorrects

Fewest incorrect attempts

Most incorrect attempts

First Attempt Hints

Fewest first-attempt hint requests

Most first-attempt hint requests

First Attempt Incorrects

Fewest first-attempt incorrect attempts

Most first-attempt incorrect attempts

Name

Lowest alphanumeric order

Highest alphanumeric order

Number of problems / KCs / students / steps

Fewest ...

Most ...

Note: The top and bottom limits are both set to
“6” upon first viewing the Performance Profiler report.

To set top and/or bottom limits:

Enter a number (zero or greater) for the top limit.

Enter a number (zero or greater) for the bottom limit.

Press Refresh Graph to update the report.

To clear a limit:

Press Clear to the right of the limit you'd
like to clear. The graph will update. You can also clear limits by
deleting the number from either or both of the limit boxes and
pressing Refresh Graph.

You can also set minimum values for filtering which rows to show. Do this by
entering values in the text fields underneath Only show rows with at least...,
and pressing Refresh Graph.

Sorting items

In the navigation bar on the left, you can choose a factor to sort by,
and specify whether that sorting should be displayed in ascending or
descending order. Selecting an option from either drop-down box will
cause the graph to update and reflect your new choice.

Displaying AFM-predicted error rate values

You also have the option of displaying steps that have no knowledge
component associated with them. To display them, check the box labeled
“include steps without a knowledge component”. As with other options,
the graph will update to reflect your choice.

Sample Selector

Sample Selector is a tool for creating and editing
samples, or groups of data you compare across—they're
not "samples" in the statistical sense, but more like filters.

By default, a single sample exists: "All Data". With the Sample
Selector, you can create new samples to organize your data.

You can use samples to:

Compare across conditions

Narrow the scope of data analysis to a specific time range,
set of students, problem category, or unit of a curriculum (for example)

A sample is composed of one or more filters, specific
conditions that narrow down your sample.

Creating a sample

The general process for creating a sample is to:

Add a filter from the categories at the left to the composition
area at the right

Modify the filter to select the subset of data you're interested
in, saving it when done

View the sample preview table to see the effect of adding your filter,
making sure you don't have an empty set (ie, a filter or combination
of filters that exclude all transactions).

Name and describe the sample

Decide whether to share the sample with others who can view the
dataset

Save the sample

The effect of multiple filters

DataShop interprets each filter after the first as an additional
restriction on the data that is included in the sample. This is also known
as a logical "AND". You can see the results of multiple filters in the
sample preview as soon as all filters are "saved".