As Stack Overflow is one of the most prominent Microsoft .NET-created sites for software developers in the world, we feel that it’s part of our mission to help lead fellow .NET developers — and the most effective way to do that is by contributing some of the code that we use to build Stack Overflow and Stack Exchange back to the greater .NET developer community as reusable open source packages. This isn’t just a fundamental part of every Stack Exchange developer’s “be more awesome” plan, it’s an explicit goal embedded in the very DNA of the company.

In fact, over the last few years, we’ve contributed a number of useful open source projects back to the world:

On Stack Exchange, you can log in with any OpenID provider and any OAuth 2.0 provider, including Google, Facebook, Yahoo, and so on. But as of May 2011, we also issue our own credentials for those people who want to have a traditional username/password arrangement. StackID is the .NET OpenID provider we created, so we can be both an OpenID consumer (we accept all OpenIDs, as well as OAuth 2.0 where available) and an OpenID provider — that is, we issue our own OpenIDs that are valid on any website that accepts an OpenID. This in turn is based on the excellent work of the open source dotNetOpenAuth library.

Here at Stack Exchange, Performance is a Feature, and we found the absolute best way to emphasize our shared family value of performance is to keep webapp performance numbers front and center in every .NET developer’s web browser. Yes, even in production. If you are a developer you’ll see a little number in the upper right hand corner on every single Stack Exchange page you load — that’s how long it took to render the page. And it’s a one click operation to drill down, two clicks to take those performance numbers and share them with blame them on someone else on the team. It’s a wonderful system that I can’t recommend highly enough to every .NET developer who works on a webapp. If that’s you, go download it. Now. Remember, we use .NET partly because it really is blindingly fast, but all it takes is a few lines of errant code to throw all those performance benefits (and more) in the toilet. So download and use MiniProfiler to make sure your fast code stays fast!

Markdown is, of course, the simple plain-text-alike markup language that we use throughout Stack Exchange for formatting. But we have to convert it from “raw” Markdown to “cooked” HTML and that takes code. We provide two implementations, for server-side C# Markdown use MarkdownSharp and for client-side JavaScript real time Markdown preview use PageDown. I did a lot of the work on MarkdownSharp, but Ben did most everything on PageDown and is now officially maintaining both of these.

Since this gets asked all the time, yes, it is legal to mix HTML of any kind within Markdown. MarkdownSharp and PageDown don’t do any cleanup of the HTML, they only guarantee that valid Markdown will be converted to valid HTML for display purposes. You must bolt on your own HTML sanitization to taste. If you’re looking for basics, start with this C# sanitization routine and this tag balancing routine. They are mostly loops and regular expressions, so trivially translatable to most languages.

Redis is our in-memory key-value store of choice. We started out using it just a little, but now it’s become an absolutely critical and totally indispensable part of our infrastructure, much like HAProxy. We use Booksleeve for pipelined, asynchronous, multiplexed and thread-safe access to Redis via our C# code. Performance here is beyond critical, as we talk to Redis from every web server constantly, and we’ve been through several passes of refinements and improvements already. If you, too, need an in-memory key-value store for your .NET webapp, consider the Redis and Booksleeve combination we use. Works great for us!

Everything contributed to Stack Exchange is under a Creative Commons license. Stack Exchange Data Explorer is the open source .NET tool that we built so anyone can browse and analyze our creative commons data via standard SQL, at data.stackexchange.com. So if you’re looking for a highly flexible, general front end to a bunch of SQL data, SEDE is your huckleberry. For more, see the blog entries we wrote about it.

This is technically something Marc Gravell created before he joined Stack Exchange, but we use protobuf-net extensively (and AFAIK exclusively) for high performance, compact serialization of .NET objects before storing them — and I daresay that our heavy use has driven the current version of Protobuf-net to be at least 3x as awesome as it would otherwise be. I don’t think you’ll find a faster and more elegant .NET serialization library in the world.

15 Comments

Don’t knock Booksleve just because of the language. It’s an awesome implementation to study if you’re developing something similar. Getting hundreds of independent transactions working over a single connection without deadlock is frigging *hard*.

I’ve used pagedown myself twice, once on a work project and once on a home project (hint hint: it powers my link). I created a django-based wrapper for pagedown, but on the back-end I used python’s Markdown instead, with mark_safe on (so it does deliberately strip out html tags) :)

Anyway, the point I was trying to make is it is one of the best written pieces of Javascript I’ve seen anywhere.

On that basis, and the fact I’m currently slowly picking up MVC, I will be sure to try Dapper instead of Entity Framework.

It allows you to put all of your projects under one “organization” so that they are easier to find. That and I think it’s easier to fork and submit pull requests through Bitbucket than it is through Google Code.

Thank you very much. Some of these tools will help me be a much better developer than my colleagues. LOL- But, no – seriously. Thank you for this and the free shirts I get some times. ! You guys are awesome.