Over the past couple of days, I have noticed that Stack Overflow is faster than ever. This is severely affecting my thumb twiddling time. When am I supposed to practice my thumb twiddling if I don't have to wait for the pages to load?? Please think of the thumb twiddlers when considering future performance enhancements!

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
If this question can be reworded to fit the rules in the help center, please edit the question.

1

Um... since you're endorsing [always-friday-in-iceland], would it be possible to remove the synonym from [fun]? pretty please? I know how to get around it with chain synonyms, but that only works once, and then upon editing, you have to manually retag it every time.
–
waiwai933Mar 25 '11 at 1:49

36

You could buy and use a 56kbit modem, that will bring back the slow performance.
–
HoLyVieRMar 25 '11 at 2:14

Wait... wait. Since when did the always-friday-in-iceland synonym get deleted?
–
uɐɯsO uɐɥʇɐNMar 25 '11 at 3:14

1

@George It didn't. However, before that synonym was created, a few synonyms to [afii] were created, such as [always-hydrogen-on-friday]. The synonym system doesn't chain retag, so if you use that tag, you get [afii]. However, when the post is edited again... the synonym system sees [afii] and retags it to [fun].
–
waiwai933Mar 25 '11 at 3:23

3 Answers
3

We looked closely at performance over the last few weeks, particularly in the last 4 or 5 business days.

we reduced the total number of queries happening on a lot of pages (question show, question lists, user page, etc.)

we deployed a page that shows us our worst query offenders in CPU time, total duration, reads, writes, etc. and optimized the stuff at the top of the lists

we improved the way we call LINQ to SQL and moved to pre-compiled LINQ queries in certain hot code paths. This seems to have gotten much faster in .NET 4.0, we saw 8x gains over plain vanilla LINQ and 2x over SQL-queries-as-LINQ-returns.

we reduced the number of columns we return in some hot code paths, where we had effectively select * from users and posts.

we were getting sloppy about the way we handled search engines, which was causing a lot of undue load -- Google's crawler is now indexing us at, and I am not making this up, 10 requests per second which is the maximum. This is confirmed in Google Webmaster Tools. Google doesn't need all the same work done per page that, say, a user does -- and when Google hits thousands of pages in a few minutes, that can kick off a lot of background work, such as rebuilding related questions. Not expensive by itself, but when multiplied by a hundred at once.. can be quite painful.

we optimized the way we retrieve site settings and configuration, which occurs sometimes hundreds of times per page.

we had some shared lock blocking and other general inefficiencies in our Redis C# library which was causing Redis (network level cache) to be slower and more blocking that it should be. This has been rewritten to be faster and block far less.

all our database calls were not quite going through the same central hub (almost, not quite) causing too many database connections in some circumstances. This has been correctly centralized now.

lots of general profiling with .NET profiling tools to identify places where we had gotten sloppy and were doing something inefficient.

we are restructuring and reorganizing our JavaScript to make the delineation between "only anonymous users need this" and "registered users need this" clear, so as to reduce the footprint of our site to an absolute minimum for anonymous users. This also significantly improves page load time (perceived and actual) for registered users as well.

You guys rock. Not only do you make stuff faster, you also give us the details :)
–
BenjolMar 25 '11 at 20:53

28

To me, disclosing the technical details about how you run and optimize your sites is one of the greatest services to the community that you guys are doing. Please never ever give it up, the totality of all these contributions already is a treasure trove for everyone interested in running a high-traffic site. (That said, this should probably go on the serverfault blog, no? ... or webmasters?)
–
PëkkaMar 28 '11 at 22:30

May I suggest one more improvement? Maybe shrinking whitespace via each html page would help with page load times and network performance. View the Source on your html page. Compressing and removing ALL white space outside the actual content of the html would help increase network performance.
–
ScottMar 29 '11 at 15:57

@Scott: I don't expect that would improve performance significantly, and since SE started as (AFAIK) a programmer help site, it makes sense to have the HTML be programmer friendly.
–
Mooing DuckMar 9 '12 at 19:37

Some congratulations are mandatory here, I always know when my connection goes off or is being sloppy: I just type "stackoverflow.com" and see how quick the answer is :)
–
Marco A.May 1 '14 at 14:20

yours is caused by bits pouring out of broken cables (bitstream liquifaction - awesome duuuuude!!). For the rest of us it is just crap peering/interconnections by TelstraClear.
–
slugsterMar 25 '11 at 7:03

1

People in New Zealand don't need the internet because they've got all that outdoors to play in. If you're in the country that invented bungie jumping and zorbing, what do you need with a computer? :)
–
SpudleyMar 29 '11 at 12:39