Custom Fields

A custom field is a new column you define for annotating transaction data.
DataShop currently supports adding and modifying custom fields at the transaction level.

You can add custom fields when logging or importing data, or after a dataset exists
(using the Custom Fields page and/or web services).

Many tutors have been instrumented to record custom fields while logging.
Some examples include a field that captures the time of each tutor response
to the millisecond; a field noting the agent that took the action in a multi-agent system;
and a field recording a categorization of the problem the student is working on.
Data mining tools can be used to generate new information about an existing dataset, and the
output of those programs can be stored in custom fields without having to create a new
dataset.

You can add or modify a custom field's metadata from the Custom Fields page (Dataset Info > Custom Fields),
but to set the data in that custom field, you need to use web
services, which is a way to interact with DataShop through a program you write.

Permissions

A custom field has an owner, the user who created it. Users who have edit or
admin permission for a project can create custom fields for a dataset in it.
Only the owner or a DataShop administrator can delete or modify the custom field.
Only DataShop administrators can delete custom fields that were logged with the data.

Custom Field Metadata

The following fields describe a custom field:

name—descriptive name for the new custom field. Must be unique across all custom fields
for the dataset. Must be no more than 255 characters.

description—description for the new custom field. Must be no more than 500 characters.

level—the level of aggregation that the custom field describes. Currently, the only accepted value
is transaction. Future versions may support other levels such as step or student.
Cannot be modified later.

Data Types

A custom field value is classified as one or more of the following data types assigned internally by DataShop:

The Custom Fields page indicates the types of custom fields, what percentage of those custom fields
fall into the aforementioned categories, and what percentage of transactions are associated with each
custom field.

Caveat: Very large custom fields may cause unexpected behavior in some applications. Excel correctly handles
exports with very large custom field values if you import the text from Excel. Other text editors may incorrectly wrap the
text values if they become too large while programs like vim, jEdit, and Notepad++ correctly handle the maximum lengths. Additionally,
when viewing custom fields in the web interface, the values are truncated to 255 characters to prevent issues with browsers. To
get the full custom field value, use the transaction export feature.

Sample Selector

Sample Selector is a tool for creating and editing
samples, or groups of data you compare across—they're
not "samples" in the statistical sense, but more like filters.

By default, a single sample exists: "All Data". With the Sample
Selector, you can create new samples to organize your data.

You can use samples to:

Compare across conditions

Narrow the scope of data analysis to a specific time range,
set of students, problem category, or unit of a curriculum (for example)

A sample is composed of one or more filters, specific
conditions that narrow down your sample.

Creating a sample

The general process for creating a sample is to:

Add a filter from the categories at the left to the composition
area at the right

Modify the filter to select the subset of data you're interested
in, saving it when done

View the sample preview table to see the effect of adding your filter,
making sure you don't have an empty set (ie, a filter or combination
of filters that exclude all transactions).

Name and describe the sample

Decide whether to share the sample with others who can view the
dataset

Save the sample

The effect of multiple filters

DataShop interprets each filter after the first as an additional
restriction on the data that is included in the sample. This is also known
as a logical "AND". You can see the results of multiple filters in the
sample preview as soon as all filters are "saved".