Data sharing options: Event Data

Event data is a record of a single event at a particular moment in time. An event consists of core attributes such as a event_name and event_type, and can be annotated with many additional attributes to provide more context. For example, Canvas captures detailed events when a user traverses a Canvas account or course.

Canvas collects numerous data points and emits some of them via its Live Events Data Service. The Live Events user interface displays a list of available event types that the customer could choose from depending on their needs. There are two formats of event data available for Canvas customers: Caliper v1p1 Basic Implementation, and Canvas Live Events Raw (Alpha). The event type and format matters when you want to answer a specific business need.

This document explains which tasks each data format is most suitable for, how to configure Canvas customer account to receive event data, and how other Canvas customers use such data.

Overview: Event Data

Event data is useful when you need to filter data based on arbitrary characteristics—such as data queries that are unique to customer external data warehouse application, learning analytics, histograms, etc. Canvas emits numerous events with a list of attributes attached to each event. For example, individual user logged_in and logged_out sessions, assignment submission details, and grade_changed transactions.

Event data is not aggregated over time; therefore, it should not be used as a single source for your data warehouse. Use it in conjunction with Canvas Data Extracts and Canvas APIs to ensure integrity of your LMS analytics dataset.

Event data is not sequenced, and can only be arranged in time based order via the "event_time" attribute expressed in ISO-8601 where data and time values are formatted with the addition of millisecond precision. The format is yyyy-MM-ddTHH:mm:ss.SSSZ where ‘T’ separates the date from the time while ‘Z’ indicates that the time is set to UTC.

Events are delivered in a "best-effort" fashion. In order to not slow down web requests, events are sent asynchronously. That means that there's a window where an event may happen, but the process responsible for sending the request to the queue is not able to queue it.

At this time Event data can only be sent to AWS SQS queue.

Event Data Formats

There are two types of Canvas event format available to our customers:

Each of the listed formats can help you to meet your institutions unique analytics needs.

Configuration requirements

Due to a new technology concept Live Events is currently an invite-only feature. All major required configurations are completed by the Instructure engineering team based on requests submitted by a client CSM.

As soon as configuration setup is complete, the navigation menu in the clients Canvas account will include a new option that reads: “Canvas Data Portal.”

The Canvas Data Portal will provide access to Canvas Data with data extracts, user management, and API credentials as well as Live Events Subscription - Caliper ONLY: where you can add a new subscription destination (SQS URL) and select the type of events you would like to receive.

Things to consider:

Not all event types are listed in the UI of the subscription. The format you are receiving events in will change the amount of event types you receive.

Please refer to the Event Type by Format table for the list of available event types for each format.

eventTime for Canvas based events does not provide millisecond precision and is expressed in the following format, e.g. "eventTime": "2018-10-01T21:22:11.000Z".

eventTime for Quizzes.Next does provide millisecond precision and expressed in the following format: "event_time": "2018-10-02T17:10:32.881Z"

Roadmap considerations

The following features are currently considered for Live Events services :

User Interface for an Administrator to subscribe to specific events for Live Events and Caliper formats

Ability to subscribe to specific events based on the producer ( User generated vs System generated )

Ability to receive data via https endpoint

Enable mobile asset_accessed events for the asset_accessed events available for web app

Additional request table data added to body of the events : course_account_id, URL of the accessed object, IP address of the event producer

Event sequencing : Include milliseconds in the Canvas "event_time" timestamp

asset_accessed event : type, subtype and triggers

How the Events are Used

Business Needs:

To determine ways in which you can make use of different types of events requires that you ask a specific business question and seek its answer by collecting and visualizing data. Here are some of the questions our clients ask when they are considering using our Live Events service:

Assessment (Quiz) Submissions

Tracking patterns using the quizzes.next and old quizzes events will allow instructors to understand more about how students are interacting with their assessments

How much time is required to submit an assessment?

If test-taking times are flexible, when do students submit their assessments?

How much time does it take to grade an assessment after a students submits it?

How often are assessments being revised by instructors?

How often are instructors relying on external assessment content or re-using assessment content (imported via QTI) when building their assessments?

User Sessions

A session is a group of user interactions in the Canvas Learning Platform that take place within a given timeframe, e.g. a single session can contain multiple page views, submission events, discussion entries, and assignment content interaction. We can think of a session as a container for the actions a user takes in Canvas. A single user can open multiple sessions. Those sessions can occur on the same day, or over several days, weeks, or months. Sessions are limited based on time limit.

What is my average user session duration?

Number of sessions by type of canvas user.

Are there any common patterns in different user sessions that attribute to the distribution and setup of the course content? I.e. Are your course structures/content convoluted and students get lost a lot?

What is the typical time of the day your users visit Canvas by user type?

When do users typically leave Canvas by user type?

What does a user Canvas session look like and are there any similar patterns across the same types of users—e.g. users participating in the same group or course?

How often is the content viewed?

What paths are taken to reach the specific content?

Course assignments and submissions

This event data could help instructors to gather insight about the relationship between students and their assignments.

What type of assignments are typically gradable?

How long does it take to submit something that has been assigned?

What assignments present the biggest challenge, e.g. which need the most retake attempts?

Course content

This event data could help administration to assess the account course content changes on periodic basis.

When does content typically get revised?

How often is course content being copied over?

How complex is the typical course content structure?

How often does the course syllabus get revised?

How Customers and Partners use Events Data Service

Examples of how partners and customers are using our data services where events services is one of them.

Blackboard Ally

Leverages Event Data Service as a notification service for a number of objects in Canvas course, events serve as triggers for this product to access Canvas data via API calls to assemble full dataset around specific course content to produce analytics around the content accessibility level. See type of events used by Ally here.

Instructure Catalog

Uses a full payload of one of our events to track student progress thru the course module progression and displays key payload data in the catalog course progress section

Clark County School District, Las Vegas

Uses events as a notification service and Canvas Data as a main source for LMS data to provide support for multiple homegrown applications such as :

nudge - student notification services

current user activity reports

education tools: student assistance services - live chat with instructors ( track when students are available to assist with their course work)

Examples of Use Cases from Customer Interviews

Use Case

Event Type

Notes

Count of logins over time by institution or account

logged_in

Current payload for logged_in event type provides "user_account_id"

First login by instructors for new institutions (for adoption support)

logged_in

Current payload for logged_in event type provides "user_account_id"

Number of LTI launches by tool

asset_accessed : context_external_tool

Reading activity vs. assessment activity

asset_accessed, submissions events

this requires some classification to determine what constitutes "reading activity"

Measure the time between when the instructor first publishes the assignment to when a student first views it

This is looking at 801 rows between the two tables. I had to create, publish and delete a test course to catch the deleted, because I don't keep these event rows very long in the database. Partly because they are events that eventually end up in Canvas Data, partly because the program I'm working on is focused on generating tables and columns for all events and I clear them daily.

I also manually concluded my workroom (test course) to fire off and watch* the event for a completed event, backed up in the browser and reopened it setting the available state.

* I can do this (in a terminal console) as long as the queue isn't backed up, and this early in the morning there is little activity.

These events are in the queue and the database within seconds.

Not sure you can do this with the Caliper event format, especially without a course_updated event. You'd probably have to handle it with custom logic.