the results have been... Not great so far: a significantly smaller number of people are clicking through to randomly-selected questions than to the top questions, which hints that the algorithm may've been doing a better job of identifying general-interest questions across topics than some expected.

Disclaimer: This should not be taken as a slight of the community team whatsoever, nor do I think this is some cause for revolt or a boxing match as the below prose may indicate. These are just poorly applied literary tools to emphasize the drastically different approaches to the same list between two groups.

In the Red Corner, the Community Members

The goal of the hot questions should be to drive up interest in the site. The hot questions should be a lure to encourage SE network users to contribute to other content, not just do a drive-by on the hot question.

In the Blue Corner, the Community Team

The goal of the hot questions should be to drive traffic to general-interest questions. After all, the Hot Network Questions used to be more accurately named as "Popular Questions".

The list of questions from a variety of sites is in a great location screen-wise, it is readily accessible and does get a lot of eyes on it. But as with any marketing, the goal isn't just to grab eyes, it's to grab the right eyes.*

* I have nothing against left eyes. Most of my friends have left eyes too. And they are awesome. But in the context right eyes are not a geospatial thing, but rather in the 'correct' sense.

So what are the right eyes? What type of people do we want to attract to our site? What would we determine as 'success'? How can we measure that success?

Please do not limit yourself to the very narrowly scoped topic above. Think outside the box if you'd like. On every page across the network we have a nice piece of real estate for showing off the rest of the network. How can that space best be used if not on a list of questions picked by an arbitrary algorithm?

Very well reasoned, very well stated, definitely not taken as a slight.
–
Tim Post♦Feb 7 '14 at 6:21

1

I look forward to your contribution @Tim! I know you guys are working on the issue, so if Jeremy and Jarrod could be brought over to share their thoughts as the guys working on this, that'd be great. In general the community team is looking at a broad range of sites, so it would be nice to see what your goals are for tracking success, and what numbers you guys are looking at to measure it. I've run some numbers but you have access to more.
–
jmac♦Feb 7 '14 at 7:04

3

'my favorite pattern is from MetaFilter... "...anyone who bookmarks that page and says "You know, I really want to be in there; maybe I'll go back later," that's the kind of user MeFi wants to have.' (A Group Is Its Own Worst Enemy)
–
gnatFeb 7 '14 at 7:14

2

That's a very good question. I cannot provide an answer though as I'm not really sure what the answer might be. I click some hot questions that seem interesting to me but it never made me consider signing up for another site. That's partially because of a conscious decision on my part though: I'm spending enough time on SO and knowing myself, I could spend too much time on another website too easily. So sorry, it will stay just a passerby.
–
SzymonFeb 7 '14 at 7:17

4

Definitely. The current selection works for "getting people over there" but definitely not in "showing people our best".
–
RaphaelFeb 7 '14 at 9:54

5

The hot list dilutes the level of individual sites by bringing in people who are not experts or deeply knowledgeable on the topic of a specific site, but have the right to determine what are good posts by voting via the association bonus anyway. This can be seen for example by observing that simple popular questions get ridiculously upvoted by the hot-list mechanisms, which gives the OPs of the hot questions (and their often simple answers) huge rep which is unrelated to their knowledge about the topic of the site. The effect on higher-level sites, such as MathOverflow is rather negative.
–
DilatonFeb 8 '14 at 17:15

@gnat yes thanks, I have already seen this and voted correspondingly on things I agree with.
–
DilatonFeb 8 '14 at 21:49

have noticed some consistent gap between what is probably "good" for sites vs what mods think is good for sites.... this may fit into that example.... se site communities can sometimes be unwelcoming to newbies etc, this is an old issue noted in the se blogs etc.... but on the other hand is good talent being scared away? hard to say! basically there are occasional occurrences of "viral-like" questions on se that are probably good for se use but not nec welcomed by the communities because of all the influx of outsiders.... personally like viral questions a lot, very insightful usually...
–
vznFeb 11 '14 at 19:48

The the SuperUser question with the second highest upvotes, which was asked only a week ago, is a joke about a toaster icon. This is starting to make me think that the Hot Network Questions are just hiding informative questions behind entertaining ones.
–
BrianAug 13 '14 at 22:57

1

@jmac - this question (and your outstanding self-answer) has been around since February. Would it be possible to know if SE team took any of your suggestions (in the answer) to heart at all? Especially the "read only" one
–
DVKAug 22 '14 at 3:03

9 Answers
9

I actually have a slightly different opinion of what "hot" questions are good for: entertainment. When I'm bored, tired of doing actual work or waiting for something to finish running, they're almost always good for a quick - and ideally informative - diversion. This is what I've always used the homepage and per-site hot lists for, and now that network-wide hot questions are available on nearly every page it's what I use that for too.

Disclaimer: this is my own opinion, based on how I've actually used these sites for 5+ years now, and doesn't necessarily reflect the opinions or goals of SE, Inc. I don't think it's all that crazy though: let's face it, when you're actually trying to solve a problem you don't care about "hotness" - you want accuracy. So what could the point of such a feature be, if not some good ol' Reddit-style idle entertainment?

The hot questions should be a lure to encourage SE network users to contribute to other content, not just do a drive-by on the hot question.

That's a good goal, but... I can't help but think "hot" lists are a terrible way of going after it. Let's face it: these aren't tailored towards anyone's interests; the only way they're encouraging answers is by just shoving questions in front of so many eyeballs that the probability of someone with knowledge of the topic sees it by chance. Don't get me wrong: a big part of the SE 2.0 model for site creation relies on folks having a wide range of interests, and jumping at the chance to participate in sites dedicated to those interests when they emerge. I have no doubt that pervasive, network-wide "hot" lists help in this goal, but calling that the primary purpose is akin to saying the primary purpose of convenience stores is to fund The March of Dimes.

10 reasons why dumb content is popular (and one crazy trick for mitigating this problem)

One more thing... Some folks love to complain about how terrible questions often end up being popular, and how poorly they reflect on the sites they crop up on. I agree: popularity often indicates a certain "lowest common denominator" focus. We've known that for years: that's why new users can't vote, why voting and close-voting are separate systems with separate privilege levels, why we struggled for years to kick embarrassing "what's your favorite..." questions out of Stack Overflow's Greatest Hits.

Guess what: if your site is full of crappy questions, your site sucks - even if they're not highly-ranked by your own users, folks are finding them via Google, and that's where the vast majority of your readers are coming from. You can work to fix that - as painful as that process is - or you can bury your head in the sand and blame it on all of those stupid people from elsewhere. If you think "hot" questions are a serious problem for your site's quality, then you're already ignoring a much bigger problem. Blaming someone else is easy and fun for the whole family - but it doesn't fix anything.

ignoring the signal is just what happens now. Go to any question that's currently on top of the hot list, find boring, repetitive, useless answers, vote them down and see what happens
–
gnatFeb 8 '14 at 21:23

4

...oh and drop your diamond while you're doing that. Things look oh so simple when you have a power to unilaterally delete any content you dislike. Try acting as us regular users have to... Vote down, comment, if you wanna flag for deletion, create a test account and flag from it, just don't cheat by handling your own flag, let a not involved mod act on it
–
gnatFeb 8 '14 at 21:42

2

@Shog9 I am curious about the 'stupid people form elsewhere' - would it be possible to get any stats from hot questions in the recent past to compare how many of the votes came from people with a reputation > 200 on the site and from people with a reputation less than 200 on the site (or you could do it based on # of answers, or days of activity - all interesting metrics)? superuser.com/questions/712551workplace.stackexchange.com/questions/18119 - I'm sure other good example can be found, those are just ones sitting at the top currently and have been an issue recently.
–
MichaelTFeb 8 '14 at 21:46

5

I'm confused, @gnat: you're responding to an answer wherein I describe how for years I've used the "hot question" features (which does not involve deletion or even voting), and you're suggesting that I'm not getting the proper experience because "regular users" do lots of down-voting, commenting and flagging? Sorry man - I've spent more time on these sites as a "regular user" than you have, and flagging has never been a significant part of my workflow; if that's where your frustration is originating, then I think I may finally see the problem here.
–
Shog9♦Feb 8 '14 at 21:59

5

@Shog9 trying to separate people who are committed to the community and not - those who showed up with just an association bonus to up vote and those who have spent some time on the site. The 200 is an approximate threshold for the two, though doing a fine grain "every 100 rep" would work as well. I am very curious about the voting patterns compared to the behavior of the core group (see Shirky, things to accept #2 & #3 / design for #3)
–
MichaelTFeb 8 '14 at 22:06

@Shog9 is my point so hard to understand? To start with, I am speaking about boring / useless answers, not about wrong ones. And I am not talking about question being boring here, but about hotness formula that stuffs these answers in as if these indicate popularity. And no, 20K are useless here because the way how broken formula rapes the sites, would require three 20Kers to keep 24x7 watch over every bit of garbage brought in from hot list lemmings
–
gnatFeb 9 '14 at 19:13

3

You're talking about stuff you don't personally like then, @gnat. Which is a problem for these discussions, since - as I told Michael yesterday - y'all keep coming up with examples that are very nearly universally-loved. Being in the minority sucks, and your frustration is further compounded because you're not even in the particular minority that the system trusts with tools. I do understand that frustration - but I'm afraid you're misdirecting it here.
–
Shog9♦Feb 9 '14 at 19:22

4

The hot questions list is global, the exact scope and tolerance for bikeshed questions is determined by each site alone. How am I supposed to remove bad questions from the hot questions list, if they are from a site where I am not active? I can go over to their meta and tell them they're doing it wrong, but aside from starting a discussions that is likely to get ugly, I don't think that would achieve very much.
–
Mad ScientistFeb 11 '14 at 14:39

4

I use the hot questions list quite frequently (or I would not complain about it so much), and I agree that entertainment is a big part of its appeal. I think it is also useful to make users aware that there is an entire network of sites, but that is more of a nice side effect.
–
Mad ScientistFeb 11 '14 at 14:43

8

@Shog9, I'm all for fixing it. Fixing takes more time than it does to get a question on the hot list though. And as any experienced editor knows, it is a lot easier to edit a question with zero answers than it is to edit one with half a dozen. The hotter a question gets, the harder it is to fix them. Yes, I could bite the bullet and make an unpopular edit for the quality of the site, but that doesn't rub me the right way and seems like a less-than-ideal solution to this problem.
–
jmac♦Feb 17 '14 at 4:21

3

@Shog, like I said in my answer, I think making hot list questions read-only on clickthrough would be a huge benefit to preventing low-quality answers. Anyone can read on them, but it takes jumping through a hoop to answer it. Those who are motivated can knock themselves out. Those wanting to share their opinion will probably not bother (or so I would hope).
–
jmac♦Feb 17 '14 at 5:42

6

I think you're crossing the beams now. If the goal is entertainment in a spare minute, then answering hot questions doesn't need to be made easy to accomplish that goal. But if you're going to shift the goalposts to suggest that the goal of putting hot questions on display for the entire site is to collect answers, then that's a totally separate issue and we need to look at data on hot list questions from first-time posters on a site: % deleted (vs. % deleted for established members), average score (vs. average score for established members), etc.
–
jmac♦Feb 17 '14 at 7:19

5

I think you are underestimating the impact of concentrated increases in traffic. Natural growth is great, because community moderation should be increasing along with visiting users. A larger community spreads out the burden and allows for community moderation economies of scale. Hot questions create flashes in the pan which are very noticeable. Compare that to SO traffic and you can see where my concern lies. How would you classify those spikes on TWP if not as flashes in the pan?
–
jmac♦Feb 18 '14 at 0:29

8

I feel like we're not having the same discussion here. As a site grows in popularity, the number of community moderators (people with privileges) increase in kind. The community becomes more resilient. Nobody is saying this is a bad thing. Do you really see no difference between a steady increase in users through natural growth, and spikes of 4-5 times normal traffic?
–
jmac♦Feb 18 '14 at 1:51

That (significant) screen real-estate should be used to strengthen communities with like-minded users

Communities should be given more control over how that space can be used and what shows up on it

If there is a need for brainless entertainment, the harm to those communities from the entertainment should be minimized

Open Source Advertising

I don't think SE should aspire to being a reddit or the like. I don't think that entertainment should be promoted over the core of creating accessible high quality answers to questions.

Right now SE uses open source advertising because they realize the value of the SE audience, and how much of a positive impact they can have on open source projects. This is a noble goal, and I'd like to think it is continued because it's working.

The same should apply to the hot questions. Like open source projects, the communities that make up the SE network require committed users with interest who can make positive contributions. This has an added benefit over the open source advertising, because it strengthens the SE network and brings more eyes and adds more value over the long run.

Hot questions currently aren't doing that. As Shog says these questions appeal to the lowest common denominator and are being used for entertainment only. The conversion rates on these new users isn't great, and while it brings a lot of eyes (and added community moderation burden), it doesn't seem to bring much long-term value to the communities that the attention is brought to.

Give Communities More Control

Like open-source advertising, let communities decide how their site should be promoted. Give at least a part of that hot questions section to an open-source advertising-esque advert for a selected community that rotates every X weeks. If communities want to promote more serious questions, or tough unanswered questions, or the most useful questions -- let them. This will grow the value of the network.

SE is based on the concept that individual communities know best what they need. That's why we have per-site metas, and per-site mods, and per-site guidelines. Not all communities may want to be promoted in the same way, and giving us the option to choose how we want to promote ourselves (or not promote ourselves as the case may be) is a natural extension of community moderation.

Minimize the Harm of Entertainment-only Questions

If you want to leave a set of entertainment questions below those adverts, that's fine. I just ask that the harm from them is minimized. If the goal is entertainment, then:

The questions should be read-only (no posting answers without a bit of effort)

Communities should be allowed to remove questions from the list at will

The community moderation team should take a part in handling the added burden

Read-only

If the goal is entertainment, that's fine. Don't let people answer when they click through to that page. The real harm is in the horrid answer quality as people add their two cents without reading the other dozen answers, or actually giving any thought in to the quality of their post. If you create a natural barrier between reading for entertainment, and being able to post an answer, that will do a lot to mitigate the harm.

As an example, you can create a read-only version of questions that require people to click through to the main page and re-find the question to post an answer. Those looking for brainless entertainment are less likely to go through the effort, and some of the worst offenders (one-line answers) can possibly be avoided by adding a single hoop to jump through.

Allow Removal from the List

Give trusted users (10k or 20k+) the ability to remove posts from the hot questions list. For whatever reason, if a question shouldn't be on there, the community should be able to purge it.

I would suggest allowing a tag to be added to posts that prevents it from showing up on the list, like a mod tag, but accessible to trusted users too.

CMs should Share the Burden

If these questions are going to be left around for entertainment value, then the Community Moderation team should park themselves in the chat rooms of all sites with a hot question. Smaller communities do not have as many mods as the big three, and oftentimes mods are not available to handle the flood of flags these questions can cause. Regardless of if the other two options are implemented, community managers should be enticed to come and help handle the burden of moderation for something SE wants. A moderation SWAT team if you will.

I don't mind entertainment being a nice time-waster, but it shouldn't hurt the communities it is promoting. That's just pouring salt in the wound.

@user, the original 3, as I understand it, are SO, SU, and MSO.
–
jmac♦Feb 17 '14 at 1:34

1

re (anti-) "entertainment" pov as (presumably) a contrast to shog9s answer: didnt really interpret his answer as asserting the purpose of the list was entertainment, only that it was his personal pov on its use, but not that it was any se "official" perspective on its purpose/meaning etc. also re bringing quality/longterm contributors, nobody so far seems to have any evidence about what actually accomplishes this very elusive/difficult (and yes, key) goal & its a lot to ask of the hot question list to serve this function! it used to be called "stickiness" yrs ago in web design....
–
vznFeb 17 '14 at 4:15

2

@vzn, while I understand that it is his personal pov, his pov tends to carry a wee bit more weight than other users which is why I wanted to address that were it to become policy for one reason or another.
–
jmac♦Feb 17 '14 at 4:19

The ideal hot questions list would present the best posts of the network that are of interest to a wider population than only experts on that subject. The hot questions are shown to a much broader audience than the source site alone, showing them very specialized posts that they won't understand is not really useful.

I think drawing some attention to the larger SE network is one important role of the hot questions list, but I don't see recruitment of new users as its main purpose. It raises awareness of other sites that the users might not have noticed otherwise, and ideally it showcases some good content, and that might lead to some users staying on the site. But the whole mechanism is not targeted enough to be very effective in that regard, the hot questions list is shown to a large population where only a tiny part is likely an expert in the subject of one specific site. I think the best ways to recruit new users need to be targeted, e.g. recommendation from a colleague or a link to the site in a place frequented by experts on that topic.

The hot questions list becomes problematic when it identifies questions that are popular, but are not really of a high quality. When the algorithm identifies a question where the community of the source sites thinks the content is not very good and maybe even a bit embarrassing for the site, there is a problem. The questions should have popular appeal, but they should still be something even experts are not embarrassed to have as a showcase for their site. Often an excellent answer alone can raise a mediocre questions to something that deserves a place in the hot questions list.

So what the algorithm should identify are questions (and their answers) that are considered good by the source community, and of interest to a much broader audience. Looking at how many users click on entries in the hot questions list is a good way to judge the broader appeal, I think. But I also think that the quality aspect needs to be considered as well. The best way I can think of to examine this would be to look at the difference in voting between users active on the site and users that arrive at the question from the hot questions list. If the source community votes very differently than the "outsiders" looking at the question, this might indicate a problem.

this looks a fairly strong point: "The questions should have popular appeal, but they should still be something even experts are not embarrassed to have as a showcase for their site." Per my experience, this is a tricky balance
–
gnatFeb 7 '14 at 15:09

1

"The hot questions list becomes problematic when it identifies questions that are popular, but are not really of a high quality." this is an elitist attitude that some ppl think they have a more valid opinion than the mass votes on questions. its like the senate overruling the house via vetoes or maybe the president overriding the senate via a veto. (and all this really has nothing to do with the hot question mechanism but instead basic se built-in voting mechanisms.) can someone compile such a list? one could point to the same list and say its evidence of small cliques unilaterally overruling
–
vznFeb 15 '14 at 0:15

2

@vzn it is natural for people who have a deep knowledge about the topic of a site (usually to be found among the local contributors to the site) to have a more valid opinion than the more curious than knowledgeable broad mass of kibitzers from the outside who are looking for fun and entertainment as Shog9 says. There is nothing "elitist" about this. Of course this is only relevant for sites who's single purpose is not just fun and entertainment by definition ...
–
DilatonFeb 16 '14 at 23:09

@Dilaton look there seems to be a lot of misconception on this issue... (1) the hot questions become hot by the local community upvoting them, and then (2) they bring in casual users once added on the hot list. it is not known how much casual users affect the voting on the question after that! there is another se-wide issue of local communities' users (incl mods) having very high clout & being able to kill questions via closing or whatever and overriding the popular vote on questions, some very high rated, and agreed the hot question list might amplify that preexisting issue...
–
vznFeb 17 '14 at 4:04

2

@vzn For #2, a start at the stats behind it can be seen in Request for hot question vote statistics. About 1/3 to 1/2 of the votes on the question come from people who are association bonus only. Though, thats a start and a glance into the information.
–
MichaelTFeb 19 '14 at 4:23

TL;DR: What else are we expected to vote down and close at Workplace and Programmers?

After discovering that one of the answers reflects my own position quite closely I wasn't going to go out on a limb but since a suggestion was made...

...Guess what: if your site is full of crappy questions, your site sucks - even if they're not highly-ranked by your own users, folks are finding them via Google, and that's where the vast majority of your readers are coming from. You can work to fix that - as painful as that process is - or you can bury your head in the sand and blame it on all of those stupid people from elsewhere. If you think "hot" questions are a serious problem for your site's quality, then you're already ignoring a much bigger problem. Blaming someone else is easy and fun for the whole family - but it doesn't fix anything.

...I would like to address that.

I participate in two sites that suffer from overly hot questions: Programmers and Workplace. In my experience, communities at these sites close inappropriate questions pretty fast, which wipes them from hot list and makes it not an issue.

Questions that really suffer from senseless over-exposure are reasonable ones, those that match site quality requirements and don't deserve closure, nor even downvotes, neither from passers by nor from community regulars.

The only remaining way to "fix" these questions, if one takes above suggestion seriously, is to cast unfair down and close votes. However pragmatic it may sound, I believe that this is a fundamentally wrong way, as it suggests bending community morale as a workaround for feature that just poorly implements specification.

Frankly, I feel quite embarrassed to see this suggested by a community manager. It feels like being told by a police officer "hey it's your fault of being robbed because you did't carry AK-47".

Oh and please spare me of extending this analogy to mumbling about dangerous districts, abandoned by law, where one would better enter armed. If a particular district of Stack Exchange City turns out broken and abandoned, there are known ways to stop the disorder and add a discipline, without letting the damage spread over and promote itself in the hot news.

It sounds like the problem is the answers then, not the questions. On Workplace, we can discuss enforcing some standards more strictly on questions where answers are about to hit the double digits. There were some answers I removed today that were just one liners.
–
jmort253Feb 15 '14 at 2:11

1

@jmort253 per my observations of questions outside of the hot list, Workplace already has strong standards and community is keen on enforcing these. I wouldn't mind to get them even stricter, but I doubt that this will solve the problem. Low quality answers are essentially impossible to predict and these bump hot questions too strong and too fast, there should be like 24x7 watch of the moderators / trusted users if we try to handle these - maybe SO can afford it, but Workplace (and Programmers, and Code Golf, and Math, and UX etc) are way too small to reliably provide such a "surveillance"
–
gnatFeb 15 '14 at 8:14

Well, I can tell you what I hoped for when I was a regular user on a tiny site: network-wide exposure. When Biblical Hermeneutics showed up in the hot question list, I vividly recall day-dreaming that everyone who had an interest in the Bible on the network would see our little site and have a burning desire to ask and answer hermeneutics questions. It was a good feeling to know that Jack's question and the answers it had already accumulated would get some exposure. It felt like our little community had earned a collective gold star.

Now I'll admit it was a bigger deal in my mind than it was in reality. As of this writing, the question has fewer than 600 views. It's not anywhere to be found on the first page of BH's Greatest Hits. Likely we didn't get even one new user out of the deal. And we certainly didn't have any crowd control problems. But you know what? I'm still proud of our little community and that question. There aren't a lot of achievements a site can earn from the time pro tem moderators are appointed until graduation, but having a "hot question" is one of them.

Back then, it wasn't as easy to find network-wide hot questions. Now that they are listed in the sidebar, it's not uncommon for me to wander over to a site I'd forgotten even existed. (Hello, Code Golf!) Ideally, I will find a question that isn't just entertaining to read, but also sparks my interest in writing something. There are some brilliant, eloquent, and thoughtful people on our sites, so I think some cross-pollination can strengthen the whole network.

The other day, I was looking into the RPG site and trying to figure out how it got a healthy bump in visitors since the beginning of the year. As I dug, it became clear that one factor has been direct traffic from other sites on the network. Digging a bit more, I turned up a series of "hot questions" such as Is 3d6 the same as 1d18? Since you don't have to be an expert in a particular RPG system to read, appreciate, or even answer this question, it attracted considerable attention (11,377 views so far). Site traffic also spiked noticeably the week the question was asked.

It was not without pain: two answers were deleted and the voting seems disproportionate. Arguably the question is not even on topic, yet it's tied for 26th on the all-time best questions list. While the answers tend to be good, none of them demonstrate any particular RPG expertise. But the question also got two solid first answers (one from Eric Lippert and the other from someone who has provided an equally upvoted answer). All-in-all, the HQ list looks like a net positive on that site.

Speaking of rolling the dice, hot questions are a sort of gamble. They take advantage of the blockbuster business model. Every day there are dozens of new hot questions. Some of them blow up and others fizzle. Most sites consider it a great success to convert just one reader in a hundred (or even a thousand) into an occasional contributor. The hot question formula improves the odds by taking into account indicators of interest and quality signalled by regular users of the site. In other words, if lots of people on a site read, answer, and vote on posts, we can feel pretty confident that showing the question around will attract similar people.

I agree that increasing viewership to good questions and driving users to sites they may wish to join but might not naturally visit are both good goals. I would like to mention a third goal:

In the....other corner, random intermittent positive reinforcement

Studies have shown that random intermittent positive reinforcement can be more effective that purely positive reinforcement. This means that a reward that is given for good behaviour, but only sometime and to varying degrees, is more effective than a proportional reward. This is perhaps because people are always chasing the next "big score".

So what has this got to do with hot questions?

Hot questions are (one can only hope) good questions with a good answer since the upvotes are a major part of what drives them - So the positive part is taken care of.

However, it is rare to get one of your questions/answers on the "Hot questions" list, and I haven't been able to detect much of a pattern (except that the questions seem to be more easily understandable to a general audience and are high quality) - So the intermittent random portion is taken care of.

And I think we can all agree you are heavily rewarded for having a good answer/question in the hot questions list.

My experience

When I got my first answer on the hot questions list it was the most exciting Stack Exchange had been since the initial honeymoon period was over. These rare periodic large rewards are key to keeping long term expert users loving the site.

Conclusion

I'm not saying that random intermittent positive reinforcement should be the main goal of the hot questions list. But it is an additional positive effect to take into consideration.

"Hot questions are (almost) always good questions with good answers since the upvotes are a major part of what drives them - So the positive part is taken care of." -- can not confirm for Computer Science. Usually, the latest question from the category broad/subjective gets picked because those that clicked, voted and answered a lot. That may be true for more sciencey sites where the real gems are often a litte more inaccessible than average.
–
RaphaelFeb 7 '14 at 9:56

11

"Hot questions are (almost) always good questions with good answers" -- what makes you think so? I ask because per my studies, part about good answers turns out wrong, see Answers quality in hot questions for more details "Summing up, I would downvote about 101-118 of 218 answers I reviewed per above research."
–
gnatFeb 7 '14 at 10:05

1

@Raphael I would say that good, but general questions end up on the hot questions. Ones which are good, but also basic. Whenever I have looked at hot questions they certainly tend to be interesting
–
Richard TingleFeb 7 '14 at 10:06

@Raphael Also, strictly speaking it is a seperate question as to if the hot questions are actually good. This is a question regarding the objectives of the hot questions rather than how effectively it achieves those objectives
–
Richard TingleFeb 7 '14 at 10:23

1

@gnat Hot questions pick up a lot of bad answers, thats the nature of the beast. But what about the top rated answer on each question? Ultimately the long tail isn't that interesting as long as the best answer is really good
–
Richard TingleFeb 7 '14 at 10:26

@RichardTingle You are right, it's another issue. However, you state that the "good" part of the selection for positive reinforcement was taken care of; I disagree. If it were indeed not true, then we'd have negative reinforcement (because good answers get less votes than bad ones on hot questions). That would not mean that positive reinforcement is a bad goal, but it would mean we'd be farther off than you assume in your answer.
–
RaphaelFeb 7 '14 at 10:32

@Raphael True, however if the assumption that hot question questions are good or can be made good is false then the whole thing should be shut down
–
Richard TingleFeb 7 '14 at 10:36

1

the way how is phrased in the answer, sounds like you say all answers are good, not only the top ones: "good questions with good answers"
–
gnatFeb 7 '14 at 12:52

The down side of this is that people will try to post "interesting" things with the intent of making the hot list. Most of these posts are poor quality but hit the joy buttons of the "average" user and become broken windows that end up hurting the over all quality of the site.
–
ChadFeb 7 '14 at 15:07

@Chad however to get onto the hot list in the first place the home community has to upvote. Certainly more generally applicable things make the hot questions but I wouldn't call them broken windows
–
Richard TingleFeb 7 '14 at 21:04

2

@RichardTingle - That a community is willing to upvote somethe does not make it a good question: workplace.stackexchange.com/q/18119/16 That is the result of hitting the hot question list despite being a horrible question
–
ChadFeb 7 '14 at 21:48

1

If the purpose of the hot list is to feature and promote rather basic generally understandable questions just to drive traffic to the individual sites, this is most probably orthogonal to the purpose the native contributors to high or even expert-level sites assign to their community ...
–
DilatonFeb 8 '14 at 22:20

3

@RichardTingle yes, but to serve people who are just starting out learning about the corresponding topic is not the purpose of every site in the network. For example for MahOverflow this is clearly not the case. The purpose and goals such expert communities define for themself should be respected. If not, it should be made clear more explicitely that all SE sites are expected to be targetted at a general audience, and exclusively expert / research-level sites are not well supported.
–
DilatonFeb 8 '14 at 22:32

Pity the poor community members of Workplace. They already have to deal with an endless supply of questions from the clueless who think that the rest of the world owes them a job and pats on the head. These are not necessarily bad questions, but they attract awful answers. Well, maybe they are not bad questions. They are made up of grammatical sentences. They seem to narrate reality as the OP sees it. Sometimes they even offer an opportunity for a really good answer, giving the OP a dose of much-needed reality. However, for every sensible answer offered by a person who has a clue, it seems like there are two or three from the orbit of Jupiter. Does this make them bad questions? Or does it just mean that a community that has taken on the job of TWP has a whole lot of work to do.

These questions are the reality TV of Stack Exchange. Broadcasting them creates work for the community, because it attracts an extra crop of drive-by dweeb answers. So I don't think that Shog9's notion that these questions are the entire fault of the community for failing to police questions is, in fact, entirely fair.

I tend to think of a bad question as one that doesn't attract good answers, among the worst being those that encourage folks to "answer" when they have no answer. That doesn't describe everything on TWP, but you could certainly be forgiven for thinking that given some of the stuff that floats up. They're getting better about seeing the forest for the trees, IMHO.
–
Shog9♦Apr 12 '14 at 23:28

I like the hot question feature! Although the algorithm might need finetuning. (it is hard to come up with good algorithms like this.)

I find this whole topic fascinating as far as the dichotomy of group voting vs individual opinion about quality. It seems on Stack Exchange,

votes determine the quality of questions!

This is exactly as is stated in the FAQ/help documentation across sites.

Now there seems to be a lot of attitude circulating that maybe there are cases where votes do not reflect question quality eg reflected in Mad Scientist's answer. But this is a bigger issue than the hot questions list to discuss elsewhere (and I am sure that would be a very interesting discussion/debate). Is there any relevant info in meta on this subject? (have not searched much here)?

The whole design of Stack Exchange is based on the idea that votes determine question quality and that concept is even referred to in the sites-wide help pages! It's a fascinating question to ask where there is a strong discrepancy between votes and quality. But then you have to define quality in some way. Is there a quantifiable way to do that? Suggest that anyone who thinks there is some definition of quality that is not the same as votes, try to quantify that eg using the data explorer. There very well may be other interesting metrics of quality, and it could be very useful to find/experiment with them.

So jmac says in the question "think outside the box" one of my favorite invitations of all! (but alas there is not much occasion to do so on SE, because, really, honestly, it has very many boxes.) My thinking here is, maybe just a good selection algorithm needs to be devised, so the ideas:

Open the floor for experimentation and others to come up with some different metrics/algorithms for rating question quality. Maybe there could be some new consensus on a new algorithm. What if the different sites could choose different algorithms from some basic list to represent their sites hot questions? or maybe even decide on terminology eg "popular" vs "hot" or "notable" or "remarkable" or whatever! and of course [dataexplorer.se] is a very key/powerful tool/resource for that exercise.

One of the big issues it seems to me is that sites are not using the mechanism to choose their own site-wide ads much. Is this working much? It seems to be half-dead on several sites that I frequent (eg muster a few votes but not the 6 quorum to pass the ad). This is a very cool feature of SE where sites can choose how to represent themselves site-wide, and it seems to be underutilized to me. If sites are not using this mechanism, and then turn around and complain about the hot question list, that seems dissonant there.