The Exam Form Statistics in the Observer Dashboard describes the various statistics associated with every Exam Form in the Project you are working on. The user can select an Exam Form from the drop-down and view the report for the corresponding Exam Form below in the Exam Form Observer Dashboard.

On selecting an Exam Form from the drop-down, information about the Exam Form will be displayed below like the page displayed below:

The following statistics are displayed:

Exam Form Statistics:

Exam Form: Shows the name of the Exam Form selected. The Exam Form ID is mentioned in the bracket following a # sign.

Average Difficulty (Angoff): The average Angoff rating of the Questions in the Exam Form.

SEM: The Standard Error of Measurement of the Exam Form.

Icon

SEM is an estimate of error to use in interpreting a candidate’s test score.

Average Validation Rating for Test: The average validation rating for the Exam Form.

Questions selected:

Scored: The number of Questions scored in the Exam Form.

Experimental: The number of experimental Questions in the Exam Form.

Percentage Shared Among Versions: The percentage of Questions shared among the versions of an Exam Form.

Metadata (% of Test): The percentage of Metadata belonging to different Metadata contents in an Exam Form.

This page also shows the details of the ratings assigned to Questions in the Project:

Column

Description

Username

The user working on the Project.

# Question Rated

The number of Questions rated by the user.

Avg Angoff

The average Angoff ratings of the questions assigned by the user.

# Rating Outliers

The number of Questions with outlier rating on the Exam Form.

Rating Difference

The difference between the average Angoff and rating outliers.

Tracking user progress in a Standard Setting session

Icon

This table is a great tool to see which users have finished work and which still have work to do using the # Questions Rated column in the table above. If you know how many Questions are on the Exam Form, it will be easy to see which users you should ping to complete their work.

Another section included is the Validation Data Report.

This report shows the validation performed on the Questions belonging to the selected Exam Form in drop-down with the following details:

Column

Description

Seq.#

Sequence on Exam Form.

Question #

Question ID (not DB ID).

Revision Code

Displays the Revision Code number of the Question.

Blueprint ID

Displays Blueprint ID of the Question.

Question Author

Displays the Author name, Date and Time for the Question.

Users who Approved Question

Displays the users who approved that Question.

Avg. Importance

Average Importance Rating (using 0 -4 rating across all raters of current version of Question).

Avg. Criticality

Average Criticality Rating (using 0 -4 rating across all raters of current version of Question).

Avg. Frequency

Average Frequency Rating (using 0 -4 rating across all raters of current version of Question).

Avg. Angoff

Average Angoff Rating (Average Angoff Rating across all raters who rated the current version of this Question).

#Raters

The number of raters who were used in calculating the averages listed above.

Raters

All of the raters whose ratings were used in calculating the values above.