I come over so many questions by 1st time users of SO that didn't even seem to take the time to read the tour. Here's one of the worse samples, completely crooked up by the OP's misconceptions about how the site works.(They just seem to think, the next open edit field is meant to post a response for anything, missing this is meant for answers)

Shouldn't we make reading the whole tour through (with a certain expected reading time, at least these 2mins) mandatory before a question can be asked or answered by a new user, instead of just inviting them and applying that Informed badge?

It could be helpful though, to have rather the badge, to indicate and sort out the people who even care to be informed, from the mere help vampires.

"If you force someone to read something, they're less likely to" is the general sentiment whenever this is requested

Well, as @AstroCB pointed in his comment I would somehow expect such reactions. Also as soon I was posting this question I realized, that the tour feature handling is from the SE engine, and might not be customized per site.

Though I could imagine that automatically increasing the entry level proportionally to the popularity of an SE site, might be a good idea.

Just an idea came to my mind, how increasing entry levels could be pushed a bit:
Let the new users actively check (confirm) each point, before taken further. This should improve to ensure they'll read what's written there.

2 Answers
2

Abstract

It's unlikely that forcing people to look at the tour page will improve their subsequent contributions. Rather, the Informed badge is a strong signal that a user has a desire to understand the Stack Overflow community and become a productive contributor.

After 2 1/2 years, we should have plenty of data to know how well the tour is working. In order to tackle the question you asked we need to try answering the following questions:

Do users who have read through the tour page ask better questions and provide better answers than those who have not?

If the statistics show correlation between reading the tour and positive contributions, can we demonstrate causation?

Given that we have a badge for at least pretending to take the tour, we can compare the results of people who joined in 2014 and earned that badge (or any other) before posting* to those who didn't:

For those following along, this query using public data shows the method I'm using. Notice that since first posts are often deleted and since deleted posts are not represented in the public data, you'll see some odd things like people earning the Tumbleweed badge for a question before they've posted anything. In those cases, the first post was deleted. The chart above corrects for that problem by including deleted posts.

Everything to the left of the vertical line represents data on the first post a user submitted. Columns to the right represent the entire career of the user. Because the site has changed substantially over the years, I'm only considering people who created their account in 2014. rep_per is a very rough estimate of the quality of a user's contributions calculated by dividing their total reputation by the number of posts they've contributed. It's not great on an individual level since voting volume varies by tag, but it should help us gauge quality on an aggregate level.

About 5% of users read the tour page (as judged by the Informed badge) before posting. Those users initial posts are far more likely to be upvoted, less likely to be deleted and (if they ask questions) less likely to be closed. Over their entire career, Informed users are also likely to post more and those posts are more likely to be upvoted. So there's a strong link between the people who tend to do well on Stack Overflow and people who have read the tour.

That's all well and good, but compare those numbers with the 2% of the people who fill in all the autobiographical information on their profile before posting. (There is some overlap between people who earn various badges before posting. It's possible this group is a near subset of the Informed.) While they do slightly worse with their first post, they tend to contribute more and better posts over their careers. Filling in autobiographical details doesn't teach people to use the site, but it does indicate that these people did their homework on how the site operates. So it seems that while reading the tour might help people with their first post, there's a good chance people who read the tour are also conscientious, which is an important attribute to have if you want to succeed on Stack Overflow.

More striking are the outliers who edit, vote (with the help of an association bonus for being active on another site) or visit the site many consecutive days before posting. These folks do far better with their first posts and (with the exception of Enthusiasts) have much more productive careers. These rare individuals not only read the short tour page, but also have read and even interacted with actual posts on the site before posting themselves. Unfortunately, we can't force everyone to study the site in that level of detail.

Since most people earn privileges required to participate by posting, that probably explains why so few people manage to, say, vote before posting. In any case, the main concern is with people who ask before viewing the tour. Answers tend to be evaluated in relation to the question they answer, but questions must create their own context. So let's focus on first questions:

N.B.: In this case, avg_posts and rep_per only count questions. Please don't compare those numbers with the previous table.

Once again, if you poke around the site a little bit by reading the tour and/or filling in your profile, your first question tends to be better than if you jump right in. And while you don't ask more often, your overall contribution to the site will probably be more impressive than most. But actual participation in the form of answering, editing, upvoting, commenting, downvoting, reviewing, and flagging turns out to be exponentially more indicative of good asking experiences.

Comparing to the badge for reading the FAQ, Analytical, turns out to be difficult because the (now retired) badge just wasn't given out as much. So it's difficult to tell if we are seeing the effects of unusually diligent users or if the information actually helped. In any case, here's the same query for people who created an account and asked their first question in 2013, the one year in which it was possible to get both badges:

Obviously we have no way to know that people actually read the FAQ, but it seems like the effort required to click a link for each section correlated with better results than simply visiting the tour page.

Correlation does not imply causation

At this point, it's tempting to say that activities such as reading the tour, suggesting edits or answering question teach users how to ask better questions. That might be true. But it's far more likely that those activities signal people who have a reason to visit the site other than merely getting answers to their questions. (For a similar situation, consider the question of whether owning books makes your children smarter.) If so, adding more requirements won't do anything to improve question quality. Instead, it will rob us of potentially valuable signal.

"but compare those numbers with the 2% of the people who fill in all the autobiographical information on their profile before posting." be careful with that 2%, spammers are known for filling the "about me" and being awarded their autobiographers tag.
– BraiamAug 26 '15 at 1:24

@Braiam: These queries do not include deleted users, so there probably are not many spammers in that 2%.
– Jon EricsonAug 26 '15 at 14:25

The bar is supposed to be intentionally low for new users. People like Eric Lippert or [insert your favorite expert here] shouldn't have to go through a tutorial to post.

The fundamental principles behind Stack Exchange are borne out of a dislike for the forum culture, and the problems it brings. The culture that many newcomers are already familiar with. In other words, most newcomers think they already know how the site works.

You're trying to solve a cultural problem by making people read. It doesn't work that way.

Your expectation is that, if we just provide enough information, people will stop asking bad questions and posting bad answers. It's an unrealistic expectation. Either people know enough to learn the cultural norms before jumping in, or they don't. Of those who don't, some will get the smackdown, learn the norms, and become good Stackizens. The rest, well...

By the way, there's a problem with the way you're mantaining your post. Do you know what it is?

"People like Eric Lippert or [insert your favorite expert here]" ... of course not, but these kinda gals and guys are trusted on SE sites anyway. We could well apply a different behavior for these, as done with other privileges, couldn't we?
– πάντα ῥεῖSep 21 '14 at 2:45

4

When they first get here, they're a 1 rep user, just like everyone else. What do you suggest, that we maintain a white list?
– Robert HarveySep 21 '14 at 2:46

Although, frankly, what difference does it make if you just remove those bits? It still reads like a chronological series of updates whether or not the labels are there. To that end I'd say removing the labels serves no benefit, and could in fact make the post more confusing to read. (Of course, there's a real "right" way to do this, but that's sidestepping my point...)
– BoltClock♦Sep 21 '14 at 2:54

The remaining conversation/discussion would not pass muster on Stack Overflow. In general, questions and answers need to read as a coherent whole, not a timeline.
– Robert HarveySep 21 '14 at 2:54

@BoltClock: The Update/Edit labels are a really bad habit. While people do seem terribly attached to them, I regularly remove them and never get any complaints.
– Robert HarveySep 21 '14 at 2:57

1

@RobertHarvey I completely agree with you that it's naive to think that forcing to read and confirm may give any retention for people that aren't interested in the sites' policies at all, and therefore accepted. Though a scheme that a user has to agree terms and policies before using some "service" is widespread (and legit) for many commercial sites, and has a meaning there. You clearly can point the user to what they were agreeing before). Well, our armour of close reasons also restates this, but I still have the feeling that increasing entry levels could be a good idea.
– πάντα ῥεῖSep 21 '14 at 3:13

2

@RobertHarvey "By the way, there's a problem with the way you're mantaining your post. Do you know what it is?" BTW, shouldn't this have been a comment on my original post, instead of appearing in your answer?!? ;-) ...
– πάντα ῥεῖSep 21 '14 at 4:17

I was making a point that was directly relevant to your question.
– Robert HarveySep 21 '14 at 5:55