Tuesday, August 31, 2010

Low-Cost Test and Quiz Tool Comparison

I’ve recently been working with Sameer Bhatia the CEO of ProProfs as I was going through an evaluation of various online tools that support easy, fast, low-cost testing and quiz tools (disclosure). He helped me pull together the following evaluation of a few different solutions:

ClassMarker

Quia

ProProfs

Google Docs Forms Creator, and

Quibblo.

Evaluation Criteria

For the specific need I was looking at, I was looking for the following out of a solution:

Question Types/Methods: Does the tool offer the ability to create a rich testing experience by offering various question types?

Multimedia: Does the tool offer the ability to include audio and/or visuals for questions?

Data Collection: Does the tool provide detailed statistics on individual and/or group performance?

Grading: Does the tool allow for several methods of grading responses? (i.e. immediate feedback, final results with or without which questions were missed, etc.)

Download Test Results? : Can the test data be retrieved for analysis or backup?

Layout/Accessibility: Can the layout of the quiz output be changed and does it handle accessibility issues?

Security: Are there security methods in place to catch/prevent cheating, hacking, or other similar problems?

Ease of Use: Is the tool well supported and easy to learn?

ProProfs

ProProfs Quiz Maker is an easy to use quiz program with a good variety of question types and options. It’s pretty amazing that it has 1 million users, over 100,000 quizzes in its public library, and over a million quiz takers. A nice feature is the end of quiz certificates for completed quizzes that can be customized. Also, test results can be downloaded in several formats. It is good at supporting accessibility with its one question per page to make the use of a screen reader more effective. It has a free and a low-cost version. To access unlimited grade reports, a paid subscription is required.

Best Features

Very easy to use – I created a quiz in minutes.

Good variety of question types.

Good option for users with accessibility issues.

Customizable quiz completion certificates.

Relatively new to corporate quizzes and tests, but has some nice features for this kind of need.

ClassMarker

ClassMarker allows you to create simple tests/quizzes for free and seems well suited to classroom use. Instructors are able to modify score reports to provide feedback. For some of the more useful features, such as viewing score reports, an active paid account is required.

Best Features

The program allows users to set up a ‘class’ so there is a unique login and password to track users.

Final score reports can be modified to be shown as the instructor wishes.

Tests can be paused, saved, and continued at the learner’s convenience.

Variety of question types can make for a dynamic quiz.

Weaknesses

All of the more useful features require a paid account, and even score reports cannot be viewed without one.

No image/video support for questions.

Not as easy to use as ProProfs.

To see a side-by-side comparison, please see the table below.

Quia

Quia allows all users to create many types of quizzes with a large variety of question types that can also include audio and video features. The instructor can set up various security settings for each class to provide more secure grading and decrease the risk of cheating. However, I found it a bit harder to use that the other tools.

Best Features

Large variety of question types available.

All quiz results are available in the free version.

Security features designed to prevent cheating.

Weaknesses

If students are not registered for a ‘class’, then the instructor cannot view the data from the quizzes.

There is no way to choose how score reports are shown to students.

Not as easy to use as some of the other tools.

To see a side-by-side comparison, please see the table below.

Google Docs Forms

Google Docs has a built-in Form document type that allows the creation of a set of questions. This can be an interesting quiz / test tool especially if you already use Google docs. There are a fair number of question types available and it’s very easy to use. However, users are anonymous while they are taking the test/quiz and thus need to self-report who they are (through a form field). This means that cheating would be quite easy. The reality is that this is more for creating surveys than creating quizzes and tests, but definitely worth considering in some situations.

Best Features

Free and easy to use

Data can be sent right to a Google docs spreadsheet or basic summary page

Good variety of question types

Weaknesses

No ‘grading’ can be done on quizzes

No cheating deterrents

Not designed for quizzes

Google docs is not included in table below since it is not designed for quizzes

Quibblo

Quibblo is a quiz and survey system that is a bit limited in terms of question types which makes it a bit harder to create a typical quiz or test. Instead users can create the type of question they require by manipulating the instructions and fields within Quibblo.

Best Features

Free quiz designer

Weaknesses

Site geared more for social settings than academic or corporate

Lots of advertisements along the sides of quizzes- distracting

Quibblo is not included in comparison table as its not an academic or corporate quiz site.

Classmarker

Quia

ProProfs

Cost

Free (some features require purchase) $24.95 - $49.95 per year

Basic is Free. $49 per year for Educators, $199 per year for Corporations

Personal use version is free. Educator version is $2.97 per month, Business version is $16.67 per month

Yes. Final score reports can be modified to be shown as the instructor wishes. Students can also save and finish the test later.

No, students will know their scores at the end of the quiz, but the instructor cannot choose how the reports are shown to students.

Yes. Several scoring options exist along with end of quiz certificate customization.

Download Test Results?

No, however, everything is backed up on the server. It cannot be downloaded.

Yes. If students are registered to the ‘class’, then the instructor can view the grades. If there is not a ‘class’ created, then there is no way to retrieve the data.

Yes, data is downloadable in several formats.

Layout

Accessibility

This cannot be changed. Overall the layout can get a little hard to follow and can take some time to get used to.

This cannot be changed. The layout is easy to understand, and someone with visual difficulties should not have problems.

This cannot be changed (though it supports one question per page or all on one page option). Someone with visual difficulties should not have problems.

Security

Each user has a unique login and password, so the information is kept secure.

Access can be restricted to the ‘class’ only, and reduces the risk of cheating.

Users all receive a unique login and password to ensure security. Quizzes can also be protected by common password.

Ease of Use

Not as easy to use as some programs.

Not as easy to use as some programs. More time was spent viewing the tutorials than creating a quiz, and even then it was still somewhat confusing.

Very easy to use. Clear and concise instructions.

Overall

Lack of reporting in the free version is not great, but it’s low-cost for the version with score reports.

The variety of questions is great, as is the ability to add multimedia to a question. However, the ‘create a class’ function is potentially daunting, and it is not as easy to use as a whole as some programs.

A new one to add to your list, which enables easy creation of online multiple-choice quizzes and exams from PowerPoint slides:

QuizSlides - https://quizslides.com

You can use the QuizSlides platform to create quizzes that can be taken anonymously which give instant feedback to the test taker, or you can use it to create multiple-choice examinations for assessing a group of examinees, with or without negative marking. Hope you like it.

About Me

Dr. Tony Karrer works as a part-time CTO for startups and midsize software companies - helping them get product out the door and turn around technology issues. He is considered one of the top technologists in eLearning and is known for working with numerous startups including being the original CTO for eHarmony for its first four years. Dr. Karrer taught Computer Science for eleven years. He has also worked on projects for many Fortune 500 companies including Credit
Suisse, Royal Bank of Canada, Citibank, Lexus, Microsoft, Nissan,
Universal, IBM, Hewlett-Packard, Sun Microsystems, Fidelity
Investments, Symbol Technologies and SHL Systemhouse. Dr. Karrer was
valedictorian at Loyola Marymount University, attended the University
of Southern California as a Tau Beta Pi fellow, one of the top 30
engineers in the nation, and received a M.S. and Ph.D. in Computer
Science. He is a frequent speaker at industry and academic events.