Program Report

Andrew Downes

July 01, 2016 18:00

What is a Program Report?

The Program Report offers an overview of a pre-defined learning program—like compliance training or new hire training. Watch as your learners progress through courses, videos, assessments and keep up with high-level metrics like average assessment scores and competencies demonstrated. Answer common questions, such as who has or hasn’t completed parts of a learning program and share it with managers and leadership.

Who can use this feature?

User Types

Any user with access to Explore (Global Admins, Area Admins, and some Users) can edit a Program report. All users can view a Program report.

Pricing

Available on paid plans (Analyst, CLO, and Enterprise).

Expertise

This feature is recommended for intermediate and expert users.

Concepts

The program report gives you insights into all aspects of your learning program from many different angles. This section outlines some key concepts that the Program Report is based on to help you navigate.

Program

A Program is a collection of activities (like courses, classes, assessments, videos, etc.) that collectively define a specific goal.

Learning Path

A Learning Path is a series of high level sequential milestones that learners are expected to pass through over the course of the program. These should be key milestones that all learners will complete and that are normally completed in a fixed order. There is no technical limit to the number of milestones included in a path but we recommend no more than six.

Workflow

A series of tasks designed to be completed in an ideal order, but which can be completed (or partially completed) in a different order. Workflow tasks are generally much more granular than a learning path, for example a workflow might be the steps taken by a call center agent to help a customer with a particular problem. Workflow reporting looks at how many people followed the ideal workflow and surfaces other common workflows.

Assessment

Assessments are quizzes, tests, observations and other interactions that assess learners and typically involve passing/failing and scoring. You might not want to include every assessment in your Program dashboard, just the most significant ones that you want to monitor.

Competency

Competencies are skills, knowledge and abilities evidenced by learners during the course of a program. Competencies are tied to a tracked activity or activities that demonstrate competence. It’s also possible for activities that report a score to indicate a % proficiency in the competency.

Contributing System

A contributing system is a tool, product or platform that is used by learners on the program. For example an LMS, a performance observation checklist and a mobile app might be three contributing systems of a program.

Using Simple Configuration to Create a Program Report

Click Explore in the main navigation and choose Program from the report-type carousel:

You’ll use Explore to customize the report and then click Saves As New Report to create a new report with the options you’ve chosen.

Using Explore to Customize Program Reports

When using Explore to customize Program Reports, you’ll see 8 options:

Click on any of the option names to look at the settings for that option:

Activities

After clicking Activities, you’ll be able to choose which activities you want to include in your Program Report. You can choose as many different activities as you’d like, and the Program Report will display information related to the chosen activities. If you don’t select any activities, the Program Report will include all activities in your account that you have permission to see.

To select an activity, use the search bar to find an activity or click the Pick From a List button to choose an activity.

Please note: You’ll only see activities that are already in your account that you have permission to see.

If you want to deselect an activity you’ve already chosen, click the X button next to the activity’s name.

People

After clicking People, choose which people to include in your Program Report. You’ll be able to select multiple people and parts of your organization, and those people will be included in the Program. If you don’t choose anyone, this report will include everyone that you have permission to see.

To select people, use the search bar to find specific people or group names. You can also click the pick from a list button to choose specific people or groups.

Please note: You’ll only be able to choose from people that you have permission to access.

If you want to deselect a person or group you’ve already chosen, click the X button next to the person or group’s name.

You can also use the People Filter to select Group Types. If you do, only these types of group will be shown in the groups table.

Dates

After clicking Dates, you can select a time range for your report to analyze. Click on the dropdown to select Trailing or Custom.

Use Trailing to select the previous X days/weeks/months/years from today. While the range stays the same, the dates change to keep up with the current date.

Use the Custom option to choose a specific date range.

If you don’t select any dates, the report will analyze all data in the account that you have permission to see.

Card Text

From the Card Text menu, customize the text on the report:

The Title is at the top of the report on both the Dashboard and the report’s detail page.

The Summary appears at the footer of the report on the Dashboard.

The Description appears at the top of the report’s detail page under the header.

Program

Within the Program menu, you can add new steps (milestones) to your Program. Each step can be a Simple Step or an Advanced Step. A Simple Step consists of only one activity, while an Advanced Step is composed of multiple activities.

Simple Step Selection

After clicking Add Simple Step, you can search for a single Activity the same way you would in the above Activities section. You also have the option to apply a name to that activity which will appear on the Program Report's milestone tracker.

Advanced Step Selection

Adding an Advanced Step allows for two differences from a Simple Step. First, you can select from multiple possible activities instead of just one. Secondly, you can identify whether or not success is required, i.e. does the activity need to be passed, or just completed?

The option to select whether steps must be completed in order is available as well. When Steps should be completed in order is selected, the Program milestones chart will always sum to 100%. When Steps can be completed in any order is selected, the Program Milestone chart will show a distinct value between 0 and 100% for each step.

Competencies

The competency menu provides the opportunity to map certain activities in a learning program to a specific competency framework. Note that this section is not required to be completed to configure a Program Report. Selecting Add Competency displays a familiar interface in which you can select an activity to associate with the competency.

The Activity Name field allows you to customize the display name of the activity to be associated to the Competency. The Activity field allows you to search for the relevant activity to associate to the Competency. If the activities selected report a score result, that score will be used to calculate the competency on a 0.0 - 1.0 scale.

After all desired activities have been associated to the competency, clicking Save will add the Competency to the Program Report.

Assessments

Within the Assessment menu, you can select one or more assessments by their activity ID after clicking the Add Assessment button.

The Assessment Name field can be populated with the display name you prefer to be displayed in the Program Report. The Activity field allows you to search for an assessment within your data. If question data from the assessment is reported which specifies the selected activity as the parent activity id, that question data will be included as well.

Advanced Configuration

The activity, people and dates filters and report text of the Program report can be configured using simple configuration as they can for all reports. In addition, the Program Report can be further configured using Advanced Configuration. Each Program report’s configuration object has a program property which contains a program configuration object. The properties of that object are outlined in the table below.

Property

Explanation

Example

name

The name of the program

Widget 5000 Sales Training

primaryPathId

The id of the primary path for the program.

Currently the program report only displays data about one learning path.

widget-5000-sales

learningPaths

An array of learning path objects representing the paths associated with the program.

Currently the program report only displays data about one learning path as identified by the primary path id.

[
{..learning path..}
]

workflows

An array of workflow objects representing the workflow associated with the program.

[
{..workflow..},
{..workflow..},
{..workflow..}
]

competencies

An array of competency objects representing the competencies associated with the program.

Please note: if a set of milestones is not always completed in the same order, setting the 'ordered' property to ‘true’ can distort the results and lead to negative percentages for steps that are commonly skipped or completed out of order.

Workflows - Advanced Configuration

Workflows represent a series of small steps that make up a single workflow within the Program. They are configured in a similar way to paths, except that there can be any number of workflows but only one primary path.

Workflow configuration objects have the following properties:

Property

Explanation

Example

type

The object type. Always ‘workflow’.

workflow

name

Name of the workflow to be displayed.

Create New Product Order

id

Id of the workflow.

new-product-order

steps

An array of step and step collection objects representing the tasks within the workflow.

The type of session. Use “registration” if workflow attempts each have a different registration.

Use “activityId” if workflow attempts each have a different activity id.

Workflows where the learner can have multiple attempts at the workflow with the same registration and activity id are not supported.

registration

stepsRequiredForCompletion

Which steps within the workflow need to be completed in order for the workflow to be considered complete. Either “all” of the steps or just the “finalStep”.

all

finalStep

exitStep

The step, which when completed, indicates that the workflow has been completed by the learner.

Contains a step object.

{..step object..}

timeLimitSeconds

The number of seconds within which the workflow must be completed from when any step is first completed.

3600

scoreActivityId

The activity id used to determine the learner’s score for the workflow.

http://example.com/customerservice/score

Please note: The first step completed should be the first step of the workflow, but learners may deviate from that path. Their time limit starts after they complete any step, even if it is not the first.

Competencies - Advanced Configuration

Competencies represent demonstrations of competence within a program. They are configured in a similar way to paths, but visualized very differently:

A program might be expected to have many competencies but only one primary path.

Steps that make up a competency should be thought of more as criteria; there is no expectation that they will be completed in a particular order.

The score achieved when completing a competency is significant and is used to determine the learner’s proficiency in that competency. Weighting can be applied to steps to account for differences in difficulty of activities demonstrating competence.

Competency configuration objects have the following properties:

Property

Explanation

Example

type

The object type. Always ‘competency’.

competency

name

Name of the competency to be displayed.

Widget 500 Demonstrations

id

Id of the competency.

widget-5000-demo

steps

An array of step and step collection objects representing the criteria for achieving the competency.

Contains a step weights object defining the weights that should be applied to each step when calculations the competency score.

Properties of the step weights object are the names of steps and the values are the weightings. Weightings should add up to 1 in total.

{
"Articulating": 0.25,
"Investigating": 0.75
}

Steps and Step Collections - Advanced Configuration

Paths and competencies both contain arrays of steps and step collections. Step collections are used to require the learner to complete a configurable number of actions from a given list. For example a learner might have to pass 2 out of 3 quizzes in order to complete the milestone. Step collections have the following properties:

Property

Explanation

Example

type

The object type. Always ‘collection’.

collection

name

Name of the step collection. If this is a top level step, this name is displayed as a program milestone.

Complete pre-training e-learning module

completeThreshold

The number of steps within the collection that need to be completed in order for the collection to be considered complete.

2

exemptPersonCustomIds

A list of custom ids of people who are exempt from completing this step. Exempt people/steps are not counted in completion percentages.

["alice@example.com", "bob@example.com"]

exemptGroupCustomIds

A list of custom ids of groups who are exempt from completing this step. Exempt people/steps are not counted in completion percentages.

["Region: West", "Region: South"]

steps

An array of step and step collection objects (collections can be nested).