Details

Description

I think Olli and I discussed this before, quite a long time ago. I am inclined to try to implement it now, because it may be as easy as any other way of fixing MDL-18214, and this is a good idea anyway.

The idea is that we change what you have to do to create a new question. Currently, you have to choose the question type you want from the 'Add question' drop-down, with only the question name to guide you.

Instead, I propose that we have an 'Add question ...' button. This will take you to a form that lists all the question types, with a brief description. You select one, then click next. If JavaScript is enabled, this will pop-up in a JavaScript dialogue (like http://developer.yahoo.com/yui/examples/container/dialog-quickstart.html) to avoid having an extra page load.

Advantages:

Makes the choice of question type much easier for novice users.

Disadvantages:

One extra mouse click for experienced users (and an extra page load if JS is off.)

The list of question types with a one-sentence description does not fit on one page* (except on big screens) so we will need a scroll-bar. That makes the first problem worse.

I am about to attach a screen-shot. Suggestions for avoiding or minimising the disadvantages welcome.

Tim Hunt
added a comment - 24/Feb/09 4:35 PM Sorry, Olli, I want your advice again. That is the price you pay for being helpful and knowledgeable. However, I do remember you are busy at the moment, so I am not expecting an answer soon.
Anyway, when you have some time, I would be grateful if you could review this. Thanks.

This model keeps the list of the labels more easily scannable, making it more easy to find each question, while allowing users to easily view the description of the question type. Also, there is less of a danger of introducing scrolling.

Olli Savolainen
added a comment - 24/Feb/09 6:47 PM - edited At first glance it looks good, although QuestionMark's model for this was better I think
---------------------------------------------------------------------------------------------------------------
Calculated____________
Select an item to view its description
Essay_______________
OR
Matching_____________
[Description for the item selected on the left column]
....
[OK] [Cancel]
---------------------------------------------------------------------------------------------------------------
This model keeps the list of the labels more easily scannable, making it more easy to find each question, while allowing users to easily view the description of the question type. Also, there is less of a danger of introducing scrolling.

I'll give this a go tomorrow. I tried a scroll bar and it just sucks. I'll have to keep the other version when JavaScript is off, but size is mostly an issue with JavaScript on, when I want to do this in a pop-up to avoid an extra page-load.

I was wondering about making Moodle remember the last question type you created, to save a mouse click when you create a lot of the same type of questions.

Also, I was going to make double-clicking on a choice select it automatically, but I have no idea how to make that discoverable. Perhaps just mention it in a help file.

Tim Hunt
added a comment - 24/Feb/09 7:23 PM I'll give this a go tomorrow. I tried a scroll bar and it just sucks. I'll have to keep the other version when JavaScript is off, but size is mostly an issue with JavaScript on, when I want to do this in a pop-up to avoid an extra page-load.
I was wondering about making Moodle remember the last question type you created, to save a mouse click when you create a lot of the same type of questions.
Also, I was going to make double-clicking on a choice select it automatically, but I have no idea how to make that discoverable. Perhaps just mention it in a help file.

Olli Savolainen
added a comment - 24/Feb/09 7:33 PM I do hope you mean an YUI dialog when you say popup.
To clarify, the element on the left is ideally a <select> list. Probably should make sure it's not ctrl-clickable to select multiple items

Ray Lawrence
added a comment - 25/Feb/09 9:31 AM Some thoughts on this:
1. The Student / Teacher references are a real turn off to non-academic users.
2. Current interface shows all options on screen, scrolling to see them is bad news.
3. Remember last question type. Probably OK if all are visible on mext visit. Suggest 1024 x 768 as lowest target monitor res for testing.

Strictly speaking it does work, but in terms of perception it is quite a disaster. It is really hard to scan that list for any keywords - namely question names - since it is so cluttered, mostly since there is no white space or organisation of information to guide the eye.

Also the question name and the description have no space between each other. The least one could do is to add that space and to make the question name bold to at least somehow allow for scanning. I think the prototype you proposed here is much less of a disaster, even if it adds scrollbars. (The scrollbars should be in the dialog, and not in the browser window.)

But ultimately, it seems to me a necessity to not show all the descriptions at once. Even tooltips might be acceptable, but it is impossible to know without testing. The model of clicking the question type name to see the description seems the best. The only way to do this without javascript would be an iframe I guess, and I do not think that is worth it. Perhaps our prototype image's model here, plus an accordion control?

Olli Savolainen
added a comment - 25/Feb/09 9:09 PM Hi Tim,
Strictly speaking it does work, but in terms of perception it is quite a disaster. It is really hard to scan that list for any keywords - namely question names - since it is so cluttered, mostly since there is no white space or organisation of information to guide the eye.
Also the question name and the description have no space between each other. The least one could do is to add that space and to make the question name bold to at least somehow allow for scanning. I think the prototype you proposed here is much less of a disaster, even if it adds scrollbars. (The scrollbars should be in the dialog, and not in the browser window.)
But ultimately, it seems to me a necessity to not show all the descriptions at once. Even tooltips might be acceptable, but it is impossible to know without testing. The model of clicking the question type name to see the description seems the best. The only way to do this without javascript would be an iframe I guess, and I do not think that is worth it. Perhaps our prototype image's model here, plus an accordion control?
http://www.leigeber.com/2008/10/animated-javascript-accordion/

Oh, pardon me it seems that somehow all the code had not refreshed and the version I saw was.. something else. Strange indeed. It looks pretty darn good now, although I am not sure about the double click, since I ended up using it accidentally. That might be because of the clicking-around nature of my usage though

Olli Savolainen
added a comment - 25/Feb/09 9:14 PM Oh, pardon me it seems that somehow all the code had not refreshed and the version I saw was.. something else. Strange indeed. It looks pretty darn good now, although I am not sure about the double click, since I ended up using it accidentally. That might be because of the clicking-around nature of my usage though

Since you give the visual clue that each selectable element is as wide as the "select box", each selectable element should also be clickable in the entire area that gets highlighted. display:block (for the label element) usually helps for this?

I would also enlarge the description text font to match the normal 1em font size of the browser or the question type name text in the dialog. There are a lot of folks out there with bad eyesight.

Also, you are introducing back the usability issue where teachers cannot find the description type since to access it, you have to "add a question" first. As annoying it might seem, there need to be three buttons at least in the quiz editing screen, the third of which is "add description" and is a shortcut to the add description page, without the new dialog.

I would also visualle separate the description type in the dialog. If nothing else, it should be the first or the last element in the list, and preferably there should be a horizontal separator between it and the other types.

Olli Savolainen
added a comment - 25/Feb/09 9:22 PM - edited Since you give the visual clue that each selectable element is as wide as the "select box", each selectable element should also be clickable in the entire area that gets highlighted. display:block (for the label element) usually helps for this?
I would also enlarge the description text font to match the normal 1em font size of the browser or the question type name text in the dialog. There are a lot of folks out there with bad eyesight.
Also, you are introducing back the usability issue where teachers cannot find the description type since to access it, you have to "add a question" first. As annoying it might seem, there need to be three buttons at least in the quiz editing screen, the third of which is "add description" and is a shortcut to the add description page, without the new dialog.
I would also visualle separate the description type in the dialog. If nothing else, it should be the first or the last element in the list, and preferably there should be a horizontal separator between it and the other types.

I'll leave double click for now, but keep an eye out for other people getting confused - it's easy to change.

display block on label: done.

Description font size: done.

I decided to try display:none on the radio buttons with JavaScript on - I think it is an improvement.

(By the way, I tried accordion-type UI for the summary, but that was really bad, because the options lower down the list moved vertically, making them very hard to hit if you were scanning down the list.)

Tim Hunt
added a comment - 26/Feb/09 2:53 PM I'll leave double click for now, but keep an eye out for other people getting confused - it's easy to change.
display block on label: done.
Description font size: done.
I decided to try display:none on the radio buttons with JavaScript on - I think it is an improvement.
(By the way, I tried accordion-type UI for the summary, but that was really bad, because the options lower down the list moved vertically, making them very hard to hit if you were scanning down the list.)

Right, so that just leaves the discoverability of Description, which is the most significant issue.

I remember that it was an important usability issue. On the other hand, we were never completely satisfied with the separate button, because we failed to find a good button caption. Add a Description / Label was always a compromise.

Whereas, in the new interface, we have a couple of sentences to explain what a Description is, which makes it much easier for a novice user to understand what this feature is, as they discover it.

I think I agree with you that we do need to move Description to one end of the list. (Drat! I had that implemented, as you can see from the earlier screenshot. Then I thought it was unnecessary, and removed it. Now you have convinced to me put it back.)

Do you agree with my supposition that, once users have discovered the Description 'fake question type', there is not an on-going usability option with having to click an 'Add a question ...' button to create one? It might actually be easier for experienced users to have fewer similar buttons in the UI?

So my hope was, that having Description in the menu, with an explanation, was sufficient to trigger the initial discovery. Especially if it is moved to the top with a divider, which I will do in a moment..

However, I think that to really find this out, we would need more usability tests, which neither of us are really in a position to do right now.

Tim Hunt
added a comment - 26/Feb/09 3:14 PM Right, so that just leaves the discoverability of Description, which is the most significant issue.
I remember that it was an important usability issue. On the other hand, we were never completely satisfied with the separate button, because we failed to find a good button caption. Add a Description / Label was always a compromise.
Whereas, in the new interface, we have a couple of sentences to explain what a Description is, which makes it much easier for a novice user to understand what this feature is, as they discover it.
I think I agree with you that we do need to move Description to one end of the list. (Drat! I had that implemented, as you can see from the earlier screenshot. Then I thought it was unnecessary, and removed it. Now you have convinced to me put it back.)
Do you agree with my supposition that, once users have discovered the Description 'fake question type', there is not an on-going usability option with having to click an 'Add a question ...' button to create one? It might actually be easier for experienced users to have fewer similar buttons in the UI?
So my hope was, that having Description in the menu, with an explanation, was sufficient to trigger the initial discovery. Especially if it is moved to the top with a divider, which I will do in a moment..
However, I think that to really find this out, we would need more usability tests, which neither of us are really in a position to do right now.

Without the radio buttons the list of types really does not afford clicking, since there is no recognizable UI interaction element - as it is, the list does not look like a select list, either. I would keep the radio buttons.

About the descriptions: I would start each of the description with the name of the selected qtype in bold.

"The answer to each of a number of sub-question must be selected from a list of possibilities."

Is there a typo here or sth since I do not quite understand it?

I would try to avoid having references to other question types in the explanation, they should be self-explanatory. If that makes the explanation longer (two paragraphs instead of one) I do not think it is a disaster to enlarge the box a bit to accommodate that, it is currently quite small.

And if you absolutely must keep references, at least have the questions in the list in a logical order; that is, Numerical just before Calculated, Matching just before RSAM. So I would think a logical order (related types grouped together) would be something like this, possibly further tweaked by how popular each type is expected to be.

Olli Savolainen
added a comment - 26/Feb/09 8:51 PM - edited Without the radio buttons the list of types really does not afford clicking, since there is no recognizable UI interaction element - as it is, the list does not look like a select list, either. I would keep the radio buttons.
About the descriptions: I would start each of the description with the name of the selected qtype in bold.
"The answer to each of a number of sub-question must be selected from a list of possibilities."
Is there a typo here or sth since I do not quite understand it?
I would try to avoid having references to other question types in the explanation, they should be self-explanatory. If that makes the explanation longer (two paragraphs instead of one) I do not think it is a disaster to enlarge the box a bit to accommodate that, it is currently quite small.
And if you absolutely must keep references, at least have the questions in the list in a logical order; that is, Numerical just before Calculated, Matching just before RSAM. So I would think a logical order (related types grouped together) would be something like this, possibly further tweaked by how popular each type is expected to be.
Short answer
Essay
True/False
Multiple choice
Numerical
Calculated
Matching
Random short-answer matching
Embedded answers (Cloze)

Whether we have the popup now or not doesn't really have to do with the original problem that teachers cannot find the description, does it.

Neither does the question of what the question type is labeled, having a button with whatever name we have come up with so far is still better than no button.

And I do not really see a substantial problem for experienced users. It seems to me that keeping the button is clearly supported by testing. And there seems to be room for the button, though that might be language dependent.

Olli Savolainen
added a comment - 26/Feb/09 8:57 PM Whether we have the popup now or not doesn't really have to do with the original problem that teachers cannot find the description, does it.
Neither does the question of what the question type is labeled, having a button with whatever name we have come up with so far is still better than no button.
And I do not really see a substantial problem for experienced users. It seems to me that keeping the button is clearly supported by testing. And there seems to be room for the button, though that might be language dependent.

Olli Savolainen
added a comment - 26/Feb/09 10:39 PM Isn't it highly unlikely though that the defaults provided in the core would not be there?
If they can be missing, isn't it even more problematic that you refer to possibly nonexistent types and thus, possibly nonexistent explanations?
Anyway, you can still assign them ordinals and then just drop whatever is missing from the list?

Logical order: No better than before this was changed to alpabetical IMO. That order may be logical to you but (for example) with the (disparate) group I've been working with today it wouldn't have any relevance for any of them. Vote to keep alphabetical.

Ray Lawrence
added a comment - 27/Feb/09 5:10 AM Logical order: No better than before this was changed to alpabetical IMO. That order may be logical to you but (for example) with the (disparate) group I've been working with today it wouldn't have any relevance for any of them. Vote to keep alphabetical.

Disparate compared to what? Unless you state the characteristics of the group you're talking about, you are simply stating that you are with a group that you assume would think something about this. Unless you tell us what is the knowledge about the group allows you to assume this, it is impossible for anyone to evaluate the validity of your statement.

My belief that grouping related items would work starts from the assumption that in order to understand what the types are about, it helps to see them as related items; instead of having to separately process to understand 10 types, this would seemingly reduce the load to the about 4 distinct types.

Also, in case of the dropdown I would also also in favor of alphabetical order, since there is no further info available in that case it is the best to allow for quick scanning of the list, and alphabetization may help that. Also, typing the first few letters of an item works best if the list is alphabetized.

Olli Savolainen
added a comment - 27/Feb/09 11:04 PM Disparate compared to what? Unless you state the characteristics of the group you're talking about, you are simply stating that you are with a group that you assume would think something about this. Unless you tell us what is the knowledge about the group allows you to assume this, it is impossible for anyone to evaluate the validity of your statement.
My belief that grouping related items would work starts from the assumption that in order to understand what the types are about, it helps to see them as related items; instead of having to separately process to understand 10 types, this would seemingly reduce the load to the about 4 distinct types.
Also, in case of the dropdown I would also also in favor of alphabetical order, since there is no further info available in that case it is the best to allow for quick scanning of the list, and alphabetization may help that. Also, typing the first few letters of an item works best if the list is alphabetized.

Disparate compared to nothing really. Not the group itself but the members of the group. Represented were commercial, government, academic and charitable sectors (and not necessarily stereotypical representation of those sectors).

The intended usage of question and quizzes varied widely. As an example of how the proposed type of grouping is flawed consider the following: the short answer and numerical questions look identical to candidates. Numerical offers some additional options not offered by short answer where a numerical answer is required, in other cases short answer can be used quite satisfactorily. This is a different approach to grouping question types than you have taken. Neither is incorrect.

In my experience MCQs are the most used question type, but in some cases that wont be true. So putting MCQ at the top is wrong for some users. It's impossible to anticipate preferences or user associations. That's why alphabetical is best here.

"Also, typing the first few letters of an item works best if the list is alphabetized."

Well only if this introduced everywhere as the behaviour for drop-downs. Let's not create different behaviour in this area only.

Ray Lawrence
added a comment - 28/Feb/09 7:44 AM Disparate compared to nothing really. Not the group itself but the members of the group. Represented were commercial, government, academic and charitable sectors (and not necessarily stereotypical representation of those sectors).
The intended usage of question and quizzes varied widely. As an example of how the proposed type of grouping is flawed consider the following: the short answer and numerical questions look identical to candidates. Numerical offers some additional options not offered by short answer where a numerical answer is required, in other cases short answer can be used quite satisfactorily. This is a different approach to grouping question types than you have taken. Neither is incorrect.
In my experience MCQs are the most used question type, but in some cases that wont be true. So putting MCQ at the top is wrong for some users. It's impossible to anticipate preferences or user associations. That's why alphabetical is best here.
"Also, typing the first few letters of an item works best if the list is alphabetized."
Well only if this introduced everywhere as the behaviour for drop-downs. Let's not create different behaviour in this area only.

"The intended usage of question and quizzes varied widely. As an example of how the proposed type of grouping is flawed consider the following: the short answer and numerical questions look identical to candidates. Numerical offers some additional options not offered by short answer where a numerical answer is required, in other cases short answer can be used quite satisfactorily. This is a different approach to grouping question types than you have taken."

I am not proposing that the order I suggested is the optimal one, but simply that it might be good to try to group the types in some order. It does not seem to me that there are an endless number of ways to group the types, and I do not see why you think that we should not try to make the order more logical just because it might not be perfect. There is little penalty from the order not being perfect, but I do think having the types in an order that gives at least some clue of which types have a similar functionality can help learning also the subtle differences between the types.

"In my experience MCQs are the most used question type, but in some cases that wont be true. So putting MCQ at the top is wrong for some users. It's impossible to anticipate preferences or user associations. That's why alphabetical is best here. "

I do not propose a preference in the list for any specific question type. Furthermore, it is not as simple as that first questions in the list are perceived most easily, in the sense of perception the last item is probably just as easily spotted, for example. The point is merely to have a grouping based on similarity of the question types so that it is easy to see what the relationships between different types are.

Your only argument for having the types alphabetized seems to be that since we cannot know what is the good order (which is not true), we should avoid making any decision whatsoever and thus we should pick the order arbitrarily. There is little benefit from alphabetisation in such a short list.

Nevertheless, this is a minor question and I will try to reply to the bigger questions you and Tim have posted in other tracker items before going on with this.

Tim, I guess it is a corner case but how does the new UI behave when the admin has added lots of different question types, say, too many to fit the screen?

Olli Savolainen
added a comment - 01/Mar/09 5:26 AM "The intended usage of question and quizzes varied widely. As an example of how the proposed type of grouping is flawed consider the following: the short answer and numerical questions look identical to candidates. Numerical offers some additional options not offered by short answer where a numerical answer is required, in other cases short answer can be used quite satisfactorily. This is a different approach to grouping question types than you have taken."
I am not proposing that the order I suggested is the optimal one, but simply that it might be good to try to group the types in some order. It does not seem to me that there are an endless number of ways to group the types, and I do not see why you think that we should not try to make the order more logical just because it might not be perfect. There is little penalty from the order not being perfect, but I do think having the types in an order that gives at least some clue of which types have a similar functionality can help learning also the subtle differences between the types.
True/False
Multiple choice
Essay
Short answer
Numerical
Calculated
Matching
Random short-answer matching
Embedded answers (Cloze)
"In my experience MCQs are the most used question type, but in some cases that wont be true. So putting MCQ at the top is wrong for some users. It's impossible to anticipate preferences or user associations. That's why alphabetical is best here. "
I do not propose a preference in the list for any specific question type. Furthermore, it is not as simple as that first questions in the list are perceived most easily, in the sense of perception the last item is probably just as easily spotted, for example. The point is merely to have a grouping based on similarity of the question types so that it is easy to see what the relationships between different types are.
Your only argument for having the types alphabetized seems to be that since we cannot know what is the good order (which is not true), we should avoid making any decision whatsoever and thus we should pick the order arbitrarily. There is little benefit from alphabetisation in such a short list.
Nevertheless, this is a minor question and I will try to reply to the bigger questions you and Tim have posted in other tracker items before going on with this.
Tim, I guess it is a corner case but how does the new UI behave when the admin has added lots of different question types, say, too many to fit the screen?

Having just said that I won't reply before addressing more urgent matters, from the depths of my gripping flu I now scream:

Yes. Thank you. You are right, and I am wrong.

Granted that "all" the other dropdowns in Moodle are alphabetized, we have just found something to include in Moodle HIG (Human Interface Guidelines, one such as http://library.gnome.org/devel/hig-book/stable/ ) so that we need not have this discussion again.

Olli Savolainen
added a comment - 03/Mar/09 3:03 AM - edited Having just said that I won't reply before addressing more urgent matters, from the depths of my gripping flu I now scream:
Yes. Thank you. You are right, and I am wrong.
Granted that "all" the other dropdowns in Moodle are alphabetized, we have just found something to include in Moodle HIG (Human Interface Guidelines, one such as http://library.gnome.org/devel/hig-book/stable/ ) so that we need not have this discussion again.

On 800x600 there is space for 17 question types, and a default Moodle install has 10. On 1024x768 there is space for 23.

At the moment, what happens when the list gets too long is that the pop-up stops fitting on the screen. Probably we need a max-height and an overflow scroll. However, I am not going to bother for now.

Also, the list was not properly sorted. It was sorted by internal qtype name, not translated name. Just fixed that.

I am rather sympathetic to the idea of a custom ordering of questions. The main reason for not implementing it was laziness, or at least wondering whether there were not more important things to do. The only thing is, how to arrange it so that the ordering copes with arbitrary new plugins? I guess the order needs to be in the database somewhere.

OK, so we can store the order in config_plugins, and potentially make it editable using the new question type management admin page, with a sensible default order.

Tim Hunt
added a comment - 03/Mar/09 2:42 PM About what happens with lots of qtypes:
On 800x600 there is space for 17 question types, and a default Moodle install has 10. On 1024x768 there is space for 23.
At the moment, what happens when the list gets too long is that the pop-up stops fitting on the screen. Probably we need a max-height and an overflow scroll. However, I am not going to bother for now.
Also, the list was not properly sorted. It was sorted by internal qtype name, not translated name. Just fixed that.
I am rather sympathetic to the idea of a custom ordering of questions. The main reason for not implementing it was laziness, or at least wondering whether there were not more important things to do. The only thing is, how to arrange it so that the ordering copes with arbitrary new plugins? I guess the order needs to be in the database somewhere.
OK, so we can store the order in config_plugins, and potentially make it editable using the new question type management admin page, with a sensible default order.

Ray Lawrence
added a comment - 04/Mar/09 4:06 PM Olli: Hope you're feeling better soon.
Tim: OK, but did I see elsewhere that you've added an option for admins to customise the order of questions? Don't have time ot search for the issue atm.

Should just stay in bed while I am sick, it seems. I'll have to take what I said in the previous comment. The alphabetized order in dropdowns has not been shown to have been tested so it does not, as is, fit into a HIG.

Also, I guess there is no example of such lists in popups in Moodle so your previous argument about consistency is not valid. Of course this is a problem in itself - for example, there seems to be no excuse not to change the dropdowns on the course pages into popups, since they suffer similar usability problems of users not knowing the functions of all the different modules/resources. This is too big to go untested, however.

Olli Savolainen
added a comment - 05/Mar/09 12:21 AM Thanks, Ray,
Should just stay in bed while I am sick, it seems. I'll have to take what I said in the previous comment. The alphabetized order in dropdowns has not been shown to have been tested so it does not, as is, fit into a HIG.
Also, I guess there is no example of such lists in popups in Moodle so your previous argument about consistency is not valid. Of course this is a problem in itself - for example, there seems to be no excuse not to change the dropdowns on the course pages into popups, since they suffer similar usability problems of users not knowing the functions of all the different modules/resources. This is too big to go untested, however.